01:49:48,847 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:49:48,865 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3991115782882141834] 01:49:48,865 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:48,882 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:48,882 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:48,882 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:48,882 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:48,882 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:48,882 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:48,882 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:48,892 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:48,903 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4361630657696220976] 01:49:48,903 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:48,919 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:48,919 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:48,919 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:48,919 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:48,942 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:48,955 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-4905562845363159020] 01:49:48,955 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:48,962 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:48,962 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:48,969 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:48,970 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:48,972 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:48,972 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:48,974 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithMultipleShards 01:49:48,996 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-87763806] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:48,996 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithMultipleShards 01:49:49,21 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: no initial generation override, starting from 0 01:49:49,21 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:49:49,22 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@53c8bfb5 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@fb6cb89 01:49:49,22 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7e60147 01:49:49,22 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionChainWithMultipleShards, 0, 1710164989022, None)) 01:49:49,22 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164989021,0,0)) 01:49:49,22 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@fb6cb89 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2651b80a 01:49:49,22 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} started 01:49:49,23 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:49,25 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:49,27 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithMultipleShards 01:49:49,27 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:49,27 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithMultipleShards 01:49:49,28 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-1-shard-people-testTransactionChainWithMultipleShards 01:49:49,28 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:49,28 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:49,28 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:49:49,28 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2ee0105f 01:49:49,28 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:49,28 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:49,37 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:49,37 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:49,37 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-cars-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards on shard member-1-shard-cars-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1996301231] 01:49:49,37 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testTransactionChainWithMultipleShards 01:49:49,37 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionChainWithMultipleShards, persistent : true 01:49:49,38 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionChainWithMultipleShards with journal batch size 1 01:49:49,38 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-1-shard-people-testTransactionChainWithMultipleShards, persistent : true 01:49:49,38 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-1-shard-people-testTransactionChainWithMultipleShards with journal batch size 1 01:49:49,39 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1996301231] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,39 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1781035748] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,39 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-notifier#1711102099 created and ready for shard:member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:49,39 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#-583806033] 01:49:49,39 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-notifier#1134643727 created and ready for shard:member-1-shard-people-testTransactionChainWithMultipleShards 01:49:49,39 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#1223617416] 01:49:49,39 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-notifier#1711102099] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,39 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:49:49,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:49:49,39 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-people-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards on shard member-1-shard-people-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1781035748] 01:49:49,40 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:49,40 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:49,41 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:49,41 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionChainWithMultipleShards is no longer the leader 01:49:49,41 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [people] 01:49:49,41 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-people-testTransactionChainWithMultipleShards is initialized 01:49:49,41 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from null to Follower 01:49:49,41 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 01:49:49,42 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from null to Follower 01:49:49,52 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-945972893] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,53 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithMultipleShards 01:49:49,54 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#-583806033] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,54 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-notifier#1134643727] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,54 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#1223617416] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,62 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: no initial generation override, starting from 0 01:49:49,62 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:49:49,62 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1816cfd4 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@c62d134 01:49:49,63 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithMultipleShards 01:49:49,63 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:49,63 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-2-shard-people-testTransactionChainWithMultipleShards 01:49:49,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:49,81 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:49,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:49,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 01:49:49,81 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [cars] 01:49:49,81 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionChainWithMultipleShards is initialized 01:49:49,81 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from null to Follower 01:49:49,82 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 01:49:49,82 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from null to Follower 01:49:49,84 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:49:49,86 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:49,87 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:49,88 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@55ecdfba 01:49:49,89 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionChainWithMultipleShards, 0, 1710164989062, None)) 01:49:49,89 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164989061,0,0)) 01:49:49,89 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@c62d134 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@82c29ff 01:49:49,90 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2d2d604f 01:49:49,90 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} started 01:49:49,91 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:49,91 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:49,91 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testTransactionChainWithMultipleShards 01:49:49,92 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionChainWithMultipleShards, persistent : true 01:49:49,92 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-people-testTransactionChainWithMultipleShards, persistent : true 01:49:49,92 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionChainWithMultipleShards with journal batch size 1 01:49:49,92 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards#-1210114473] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,92 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards#800829735] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,93 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-people-testTransactionChainWithMultipleShards with journal batch size 1 01:49:49,93 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-notifier#1704710830 created and ready for shard:member-2-shard-people-testTransactionChainWithMultipleShards 01:49:49,93 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-1664621247] 01:49:49,93 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-notifier#1773851650] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,93 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-notifier#1704710830] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,93 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-1664621247] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,98 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-notifier#1773851650 created and ready for shard:member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:49,101 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:49,101 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:49,102 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-1386386767] 01:49:49,103 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-1386386767] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:49,109 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:49,110 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [people] 01:49:49,110 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-testTransactionChainWithMultipleShards is initialized 01:49:49,110 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:49,110 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:49,110 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:49,114 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:49,115 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:49,115 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:49,115 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 01:49:49,115 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:49,115 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithMultipleShards , received role change from null to Follower 01:49:49,116 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:49:49,116 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:49:49,116 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [cars] 01:49:49,116 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionChainWithMultipleShards is initialized 01:49:49,116 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 01:49:49,117 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:49,117 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionChainWithMultipleShards is no longer the leader 01:49:49,117 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionChainWithMultipleShards , received role change from null to Follower 01:49:49,117 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 01:49:49,117 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:49:49,117 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-2-shard-people-testTransactionChainWithMultipleShards from null to Follower 01:49:49,117 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-2-shard-cars-testTransactionChainWithMultipleShards from null to Follower 01:49:49,141 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-87763806] 01:49:49,141 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: findLocalShard : cars 01:49:49,141 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1996301231] 01:49:49,275 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:49,275 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:49:49,275 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testTransactionChainWithMultipleShards] 01:49:49,275 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 01:49:49,275 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 1 01:49:49,275 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:49,275 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Closing 01:49:49,275 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:49,275 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionChainWithMultipleShards is no longer the leader 01:49:49,275 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from Follower to Candidate 01:49:49,276 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from Follower to Candidate 01:49:49,394 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:49,394 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:49:49,394 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionChainWithMultipleShards] 01:49:49,394 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:49,394 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 1 01:49:49,394 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:49,394 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Closing 01:49:49,394 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:49:49,394 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 01:49:49,394 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from Follower to Candidate 01:49:49,394 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from Follower to Candidate 01:49:49,564 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:49:49,564 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 01:49:49,564 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 2 01:49:49,670 PM [cluster-test-shard-dispatcher-32] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:49,674 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:49:49,674 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:49,674 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 2 01:49:49,680 PM [cluster-test-shard-dispatcher-32] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:49,884 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:49:49,884 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 01:49:49,884 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 3 01:49:49,885 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:49:49,885 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:49,887 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:49,887 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards 01:49:49,887 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:49,887 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards 01:49:49,887 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:49:49,887 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:49,887 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:49,887 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:49,887 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:49:49,887 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:49,887 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:49:49,887 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-2-shard-people-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:49:49,899 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:49,899 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards 01:49:49,899 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:49,899 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards 01:49:49,899 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:49:49,899 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:49,899 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:49,899 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:49,899 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:49:49,899 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:49:49,946 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:49:49,946 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:49,947 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 3 01:49:49,947 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:49,952 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:49,952 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Cluster state: [] 01:49:49,952 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:49,952 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:49,952 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=null 01:49:49,952 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:49,952 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:49,952 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 01:49:49,955 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 01:49:49,955 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 01:49:49,955 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionChainWithMultipleShards] 01:49:49,956 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:49,956 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 01:49:49,956 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:49:49,956 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: became leader with frontend state for [] 01:49:49,956 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from Candidate to Leader 01:49:49,956 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 01:49:49,956 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:49,956 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:49,956 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from Candidate to Leader 01:49:49,960 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithMultipleShards: Closing 01:49:49,960 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testTransactionChainWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:49,961 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-cars-testTransactionChainWithMultipleShards status sync done false 01:49:49,962 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:49:49,962 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:49,964 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 01:49:49,964 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:49,964 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:49,983 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:49:50,51 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-87763806] 01:49:50,51 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: findLocalShard : people 01:49:50,51 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1781035748] 01:49:50,81 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-cars-testTransactionChainWithMultipleShards status sync done true 01:49:50,144 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:49:50,144 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 01:49:50,144 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 4 01:49:50,145 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testTransactionChainWithMultipleShards 01:49:50,147 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:50,147 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Cluster state: [] 01:49:50,147 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:50,147 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:50,147 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=null 01:49:50,147 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:50,147 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 01:49:50,147 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:49:50,154 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:49:50,154 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:49:50,154 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testTransactionChainWithMultipleShards] 01:49:50,154 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:50,154 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:49:50,154 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:49:50,154 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: became leader with frontend state for [] 01:49:50,154 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from Candidate to Leader 01:49:50,154 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 01:49:50,154 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:49:50,154 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:49:50,154 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from Candidate to Leader 01:49:50,154 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All Shards are ready - data store testTransactionChainWithMultipleShards is ready 01:49:50,156 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionChainWithMultipleShards: Closing 01:49:50,156 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 01:49:50,156 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testTransactionChainWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:49:50,156 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:49:50,156 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:49:50,156 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All Shards are ready - data store testTransactionChainWithMultipleShards is ready 01:49:50,156 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-people-testTransactionChainWithMultipleShards status sync done false 01:49:50,157 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:49:50,157 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:50,203 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionChainWithMultipleShards commit rateLimit adjusted to 131.69507394001653 01:49:50,203 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,203 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-945972893] 01:49:50,203 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:49:50,203 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-945972893] 01:49:50,203 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:49:50,204 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:49:50,204 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:49:50,204 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Readying 2 components for commit 01:49:50,204 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 01:49:50,204 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:49:50,204 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Readying transaction for shard people 01:49:50,204 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:49:50,204 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 canCommit 01:49:50,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:50,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards, primaryVersion=13] 01:49:50,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:49:50,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards, primaryVersion=13] 01:49:50,204 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards, primaryVersion=13] 01:49:50,204 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] for shard cars 01:49:50,204 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] found - creating WRITE_ONLY transaction context 01:49:50,204 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:49:50,205 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:49:50,205 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 = [ 2 ] 01:49:50,205 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction called 01:49:50,205 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 sending 2 batched modifications, ready: true 01:49:50,205 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] 01:49:50,205 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards, primaryVersion=13] 01:49:50,205 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] for shard people 01:49:50,205 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] found - creating WRITE_ONLY transaction context 01:49:50,205 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 01:49:50,205 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person 01:49:50,205 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 = [ 2 ] 01:49:50,205 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction called 01:49:50,205 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 sending 2 batched modifications, ready: true 01:49:50,205 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] 01:49:50,209 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, disabling transaction tracking 01:49:50,209 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, disabling transaction tracking 01:49:50,209 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 01:49:50,209 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 01:49:50,209 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 01:49:50,209 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:49:50,209 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 01:49:50,209 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:50,209 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:50,209 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:49:50,209 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}]]] 01:49:50,209 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 01:49:50,209 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:49:50,209 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 01:49:50,209 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 01:49:50,209 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:49:50,209 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 01:49:50,209 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}]]] 01:49:50,209 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@551dbff7 01:49:50,209 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying 2 batched modifications for Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,210 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 01:49:50,210 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 01:49:50,210 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:49:50,210 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Readying Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 of 2 operations, client version 13 01:49:50,210 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,210 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 01:49:50,210 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@f56e6 01:49:50,210 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 ready 01:49:50,210 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying 2 batched modifications for Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,210 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 01:49:50,210 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Readying Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 of 2 operations, client version 13 01:49:50,210 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,210 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 ready 01:49:50,210 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE, modificationType=WRITE}}}]} 01:49:50,212 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 01:49:50,212 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 01:49:50,212 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 01:49:50,212 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 01:49:50,213 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}]]] 01:49:50,213 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 01:49:50,213 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:49:50,213 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 01:49:50,213 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 01:49:50,213 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 1 01:49:50,213 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:50,214 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction 01:49:50,214 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1996301231)] resolved 01:49:50,214 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:49:50,214 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction 01:49:50,214 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1781035748)] resolved 01:49:50,214 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: successfully resolved all cohort actors 01:49:50,214 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 finishCanCommit 01:49:50,214 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1996301231)] 01:49:50,214 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1996301231)] 01:49:50,214 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:49:50,214 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 01:49:50,214 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}]]] 01:49:50,214 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 01:49:50,214 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:49:50,214 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 01:49:50,214 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 01:49:50,214 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 1 01:49:50,214 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:50,215 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 01:49:50,215 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-people-testTransactionChainWithMultipleShards status sync done true 01:49:50,226 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:49:50,226 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:49:50,226 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 0 01:49:50,226 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 01:49:50,226 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 01:49:50,226 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:49:50,226 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionChainWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:49:50,227 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, nanosAgo=17728904} is already disabled 01:49:50,227 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:50,227 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:50,227 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}]]] 01:49:50,227 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:49:50,227 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Can committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,227 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,227 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 validated 01:49:50,227 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:50,227 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: success 01:49:50,228 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:49:50,228 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:49:50,228 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 0 01:49:50,228 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 01:49:50,228 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 01:49:50,228 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:49:50,228 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionChainWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:49:50,228 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, nanosAgo=18827532} is already disabled 01:49:50,228 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:49:50,228 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 01:49:50,228 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}]]] 01:49:50,228 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:49:50,230 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}]]] 01:49:50,230 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 01:49:50,230 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:49:50,230 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 01:49:50,230 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 01:49:50,230 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 2 01:49:50,230 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 0 01:49:50,230 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:50,230 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:49:50,231 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 0 01:49:50,231 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 01:49:50,231 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 01:49:50,231 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:49:50,231 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionChainWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:49:50,231 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}]]] 01:49:50,231 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 01:49:50,231 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:49:50,231 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 01:49:50,231 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 01:49:50,231 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 2 01:49:50,231 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 0 01:49:50,231 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:50,231 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:49:50,231 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 0 01:49:50,231 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 01:49:50,231 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 01:49:50,231 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:49:50,231 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionChainWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:49:50,233 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:49:50,233 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1781035748)] 01:49:50,233 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1781035748)] 01:49:50,236 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 01:49:50,236 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:49:50,236 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 01:49:50,236 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:49:50,237 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:49:50,237 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:49:50,237 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 1 01:49:50,237 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 01:49:50,237 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 01:49:50,237 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} has no callback 01:49:50,237 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 3 01:49:50,237 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:50,237 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:49:50,237 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:49:50,237 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:49:50,237 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 1 01:49:50,237 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 01:49:50,237 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 01:49:50,237 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} has no callback 01:49:50,237 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:49:50,237 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:49:50,237 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:49:50,238 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Can committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,238 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,238 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 validated 01:49:50,238 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:50,238 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: success 01:49:50,239 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 1 01:49:50,239 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:49:50,239 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 1 01:49:50,239 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 01:49:50,239 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 01:49:50,239 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 1 01:49:50,239 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 3 01:49:50,239 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:49:50,239 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 1 01:49:50,239 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:49:50,239 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 01:49:50,239 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 01:49:50,239 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:49:50,239 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:49:50,240 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:49:50,240 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit returning result: true 01:49:50,241 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 commit 01:49:50,241 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 finish commit 01:49:50,241 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1996301231)] 01:49:50,241 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1996301231)] 01:49:50,241 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1781035748)] 01:49:50,241 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1781035748)] 01:49:50,245 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,245 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,246 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 candidate ready 01:49:50,246 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit - candidate rootPath: / 01:49:50,246 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,246 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,246 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit - messages: [] 01:49:50,246 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: preCommit - successfulFromPrevious: [] 01:49:50,246 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 prepared 01:49:50,246 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 candidate ready 01:49:50,246 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:50,246 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit - candidate rootPath: / 01:49:50,246 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,246 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,246 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit - messages: [] 01:49:50,246 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: preCommit - successfulFromPrevious: [] 01:49:50,246 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 prepared 01:49:50,246 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:50,246 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,246 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,246 PM [cluster-test-shard-dispatcher-23] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 176 01:49:50,246 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}] 01:49:50,246 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}] 01:49:50,246 PM [cluster-test-shard-dispatcher-25] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 180 01:49:50,246 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:50,246 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=180}] 01:49:50,246 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=180}] 01:49:50,246 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:49:50,246 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:50,246 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:50,246 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}]]] 01:49:50,246 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:49:50,246 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 01:49:50,246 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 submitted to persistence 01:49:50,246 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=180}]]] 01:49:50,246 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 submitted to persistence 01:49:50,249 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}]]] 01:49:50,249 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 01:49:50,249 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:49:50,249 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=176} 01:49:50,249 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}] 01:49:50,249 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 2 01:49:50,249 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=2, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:50,249 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=180}]]] 01:49:50,249 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 01:49:50,249 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:49:50,249 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=180} 01:49:50,249 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=180}] 01:49:50,249 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 2 01:49:50,249 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:50,251 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 2, nextIndex: 3 01:49:50,251 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 2, nextIndex: 3 01:49:50,251 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:49:50,251 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:49:50,251 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}] 01:49:50,251 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=180}] 01:49:50,254 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}] 01:49:50,254 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=180}] 01:49:50,254 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:49:50,254 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:49:50,254 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 2 01:49:50,254 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 2 01:49:50,254 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 01:49:50,254 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 01:49:50,254 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 2 data Simple{size=180} 01:49:50,254 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 2 data Simple{size=176} 01:49:50,254 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,254 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,254 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: commit - successfulFromPrevious: [] 01:49:50,254 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:49:50,254 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: commit - successfulFromPrevious: [] 01:49:50,254 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:49:50,254 PM [cluster-test-shard-dispatcher-23] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE, modificationType=WRITE}}}]} 01:49:50,254 PM [cluster-test-shard-dispatcher-25] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 01:49:50,254 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-people-testTransactionChainWithMultipleShards%231781035748$f] 01:49:50,254 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-cars-testTransactionChainWithMultipleShards%23-1996301231$e] 01:49:50,255 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#1682259534] with name member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher 01:49:50,255 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#1674546592] with name member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher 01:49:50,255 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#1674546592] 01:49:50,255 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 4 01:49:50,255 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 3 01:49:50,255 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:49:50,255 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 01:49:50,255 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 104.5 μs 01:49:50,255 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#1674546592] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:50,255 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#1682259534] 01:49:50,255 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:49:50,255 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 35.28 μs 01:49:50,255 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 01:49:50,256 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#1682259534] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:50,257 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: commit succeeded 01:49:50,257 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,257 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] for shard cars, trying to use DirectTransactionContextWrapper 01:49:50,257 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:49:50,257 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] found - trying create transaction 01:49:50,257 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] 01:49:50,257 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Queuing TransactionOperation 01:49:50,257 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] for shard people, trying to use DirectTransactionContextWrapper 01:49:50,257 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:49:50,257 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] found - trying create transaction 01:49:50,257 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] 01:49:50,257 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Queuing TransactionOperation 01:49:50,258 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Queuing TransactionOperation 01:49:50,259 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Creating transaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,259 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Creating transaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,259 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 01:49:50,259 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 01:49:50,259 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@65e3e7f9 01:49:50,259 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6268ffb1 01:49:50,260 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#-1681926440] 01:49:50,260 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#-1681926440] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:50,260 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#621092277] 01:49:50,261 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#-1681926440, transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] 01:49:50,261 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#621092277, transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] 01:49:50,262 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 01:49:50,262 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] 01:49:50,262 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 01:49:50,262 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 sending 1 batched modifications, ready: false 01:49:50,262 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#621092277)] 01:49:50,262 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#621092277)] 01:49:50,267 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 modifications completed with BatchedModificationsReply [numBatched=1] 01:49:50,267 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 ReadData operation succeeded 01:49:50,268 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] 01:49:50,268 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 sending 1 batched modifications, ready: false 01:49:50,268 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#-1681926440)] 01:49:50,268 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#-1681926440)] 01:49:50,275 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 modifications completed with BatchedModificationsReply [numBatched=1] 01:49:50,275 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 ReadData operation succeeded 01:49:50,275 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Readying 2 components for commit 01:49:50,275 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Readying transaction for shard cars 01:49:50,275 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 = [ 1 ] 01:49:50,275 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 readyTransaction called 01:49:50,275 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 sending 0 batched modifications, ready: true 01:49:50,275 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#621092277)] 01:49:50,275 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Readying transaction for shard people 01:49:50,275 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 = [ 1 ] 01:49:50,275 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 readyTransaction called 01:49:50,275 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 sending 0 batched modifications, ready: true 01:49:50,275 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#-1681926440)] 01:49:50,275 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:49:50,275 PM [main] [DEBUG] TransactionChainProxy - Tx: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 - waiting for ready futures with pending Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:49:50,275 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Queuing TransactionOperation 01:49:50,275 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Readying 1 components for commit 01:49:50,275 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Readying transaction for shard people 01:49:50,275 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Queuing TransactionOperation 01:49:50,276 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 canCommit 01:49:50,277 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,277 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,277 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: handleForwardedReadyTransaction for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,277 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: handleForwardedReadyTransaction for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,277 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, client version 13 01:49:50,277 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, client version 13 01:49:50,277 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,277 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,277 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 ready 01:49:50,277 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 ready 01:49:50,277 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}]} 01:49:50,277 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:49:50,283 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 readyTransaction 01:49:50,283 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 modifications completed with ReadyTransactionReply [version=13] 01:49:50,283 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 readyTransaction 01:49:50,283 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1781035748)] resolved 01:49:50,283 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 modifications completed with ReadyTransactionReply [version=13] 01:49:50,283 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1996301231)] resolved 01:49:50,283 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: successfully resolved all cohort actors 01:49:50,283 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 finishCanCommit 01:49:50,283 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1996301231)] 01:49:50,283 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1996301231)] 01:49:50,283 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionChainProxy - Tx: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 - previous Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 readied - proceeding to FindPrimaryShard 01:49:50,283 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] for shard people 01:49:50,283 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] found - creating WRITE_ONLY transaction context 01:49:50,283 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 executeDelete called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] 01:49:50,283 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 directCommit called 01:49:50,283 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 sending 1 batched modifications, ready: true 01:49:50,283 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] 01:49:50,287 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Can committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,287 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,287 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 validated 01:49:50,287 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:50,287 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: success 01:49:50,288 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Reusing a chained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 01:49:50,288 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 snapshot MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}] 01:49:50,288 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:49:50,288 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Readying Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 of 1 operations, client version 13 01:49:50,288 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:49:50,288 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 ready 01:49:50,288 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=DELETE}}}}}}}]} 01:49:50,288 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 is scheduled for canCommit step 01:49:50,289 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:49:50,289 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1781035748)] 01:49:50,289 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1781035748)] 01:49:50,291 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Can committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,291 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,291 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 validated 01:49:50,291 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:50,291 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: success 01:49:50,293 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:49:50,293 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit returning result: true 01:49:50,299 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 commit 01:49:50,299 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 finish commit 01:49:50,299 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1996301231)] 01:49:50,299 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1996301231)] 01:49:50,299 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1781035748)] 01:49:50,299 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1781035748)] 01:49:50,301 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,301 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,302 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,302 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,302 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 candidate ready 01:49:50,302 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit - candidate rootPath: / 01:49:50,302 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit - messages: [] 01:49:50,302 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: preCommit - successfulFromPrevious: [] 01:49:50,302 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 prepared 01:49:50,302 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:50,302 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,302 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,302 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 candidate ready 01:49:50,302 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit - candidate rootPath: / 01:49:50,302 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit - messages: [] 01:49:50,302 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: preCommit - successfulFromPrevious: [] 01:49:50,302 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 prepared 01:49:50,302 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:50,302 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,302 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,302 PM [cluster-test-shard-dispatcher-23] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 235 01:49:50,302 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}] 01:49:50,302 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}] 01:49:50,302 PM [cluster-test-shard-dispatcher-25] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 224 01:49:50,302 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:49:50,302 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 validated 01:49:50,302 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:49:50,302 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: success 01:49:50,302 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:49:50,302 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:49:50,302 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:50,303 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 candidate ready 01:49:50,303 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: canCommit - candidate rootPath: / 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: canCommit - messages: [] 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: preCommit - successfulFromPrevious: [] 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 prepared 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:49:50,303 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=3, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}]]] 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 scheduled for commit step 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=224}] 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=224}] 01:49:50,303 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 submitted to persistence 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: false 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 submitted to persistence 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 221 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=221}] 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=221}] 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 3 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=224}], SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=221}]]] 01:49:50,303 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 submitted to persistence 01:49:50,304 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}] 01:49:50,307 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=224}] 01:49:50,308 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=221}] 01:49:50,308 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=3, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}]]] 01:49:50,308 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 01:49:50,308 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:49:50,308 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=235} 01:49:50,309 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}] 01:49:50,309 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 3 01:49:50,309 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 2 01:49:50,309 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:50,309 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 3 01:49:50,309 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 2 01:49:50,309 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 01:49:50,309 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 2 data Simple{size=176} 01:49:50,309 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1020149008] with name member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher 01:49:50,310 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1020149008] 01:49:50,310 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:49:50,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 3 01:49:50,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=224}], SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=221}]]] 01:49:50,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 01:49:50,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:49:50,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=224} 01:49:50,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=224}] 01:49:50,310 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 144.2 μs 01:49:50,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=221} 01:49:50,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=221}] 01:49:50,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 4 01:49:50,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 2 01:49:50,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:49:50,310 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1020149008] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:50,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 4 01:49:50,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 2 01:49:50,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 01:49:50,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 2 data Simple{size=180} 01:49:50,310 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:49:50,311 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#1736109855] with name member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher 01:49:50,311 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 4 01:49:50,311 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#1736109855] 01:49:50,311 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:49:50,311 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 92.11 μs 01:49:50,311 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#1736109855] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:50,312 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}] 01:49:50,312 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 01:49:50,312 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=224}] 01:49:50,312 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=221}] 01:49:50,312 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 01:49:50,313 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 4, nextIndex: 5 01:49:50,313 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 3, nextIndex: 4 01:49:50,313 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 2 01:49:50,313 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 01:49:50,313 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 3 01:49:50,313 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 01:49:50,313 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 3 01:49:50,313 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 01:49:50,313 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=224} 01:49:50,313 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,313 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=235} 01:49:50,313 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,313 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: commit - successfulFromPrevious: [] 01:49:50,313 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 01:49:50,313 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: commit - successfulFromPrevious: [] 01:49:50,313 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 01:49:50,313 PM [cluster-test-shard-dispatcher-23] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}]} 01:49:50,313 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-people-testTransactionChainWithMultipleShards%231781035748$s] 01:49:50,313 PM [cluster-test-shard-dispatcher-25] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:49:50,313 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-cars-testTransactionChainWithMultipleShards%23-1996301231$r] 01:49:50,314 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 4 01:49:50,314 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 01:49:50,314 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 4 data Simple{size=221} 01:49:50,314 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:49:50,314 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: commit - successfulFromPrevious: [] 01:49:50,314 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 01:49:50,314 PM [cluster-test-shard-dispatcher-23] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, modificationType=DISAPPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=DELETE, modificationType=DELETE}}}}}}}]} 01:49:50,314 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-people-testTransactionChainWithMultipleShards$p] 01:49:50,314 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 2 for term 3 01:49:50,314 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 4 01:49:50,314 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:49:50,314 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:49:50,314 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=3 01:49:50,314 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 01:49:50,314 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:49:50,314 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 59.04 μs 01:49:50,314 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:49:50,314 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 28.20 μs 01:49:50,314 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:49:50,315 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 32.24 μs 01:49:50,318 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 3 01:49:50,318 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 01:49:50,318 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 3 01:49:50,318 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 01:49:50,318 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=235} 01:49:50,319 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,319 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 2 for term 3 01:49:50,319 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=3 01:49:50,319 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 4 01:49:50,319 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 01:49:50,319 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 3 01:49:50,319 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 01:49:50,319 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=224} 01:49:50,319 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:49:50,320 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 4 01:49:50,320 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 01:49:50,320 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 4 data Simple{size=221} 01:49:50,320 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:49:50,320 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: commit succeeded 01:49:50,320 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 modifications completed with CommitTransactionReply [version=13] 01:49:50,320 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:49:50,321 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 62.91 μs 01:49:50,321 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:49:50,321 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 34.57 μs 01:49:50,321 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:49:50,321 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 39.87 μs 01:49:50,321 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 4 01:49:50,321 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 01:49:50,321 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 canCommit 01:49:50,321 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 successfully completed direct commit 01:49:50,322 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 01:49:50,322 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] for shard cars, trying to use DirectTransactionContextWrapper 01:49:50,322 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:49:50,322 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] found - trying create transaction 01:49:50,322 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] 01:49:50,322 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 01:49:50,327 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 01:49:50,327 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 01:49:50,327 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Creating transaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 01:49:50,327 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:0_6#745588837] 01:49:50,330 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:0_6#745588837, transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0, version=13] 01:49:50,330 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:49:50,330 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:0_6#745588837)] 01:49:50,334 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 ReadData operation succeeded 01:49:50,338 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] for shard people, trying to use DirectTransactionContextWrapper 01:49:50,338 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:49:50,338 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] found - trying create transaction 01:49:50,338 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] 01:49:50,338 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 01:49:50,339 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Creating transaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 01:49:50,340 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:0_7#-1999383556] 01:49:50,341 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:0_7#-1999383556, transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0, version=13] 01:49:50,341 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] 01:49:50,341 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:0_7#-1999383556)] 01:49:50,344 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 ReadData operation succeeded 01:49:50,345 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:49:50,345 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:50,345 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testTransactionChainWithMultipleShards 01:49:50,346 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onShutDown 01:49:50,346 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithMultipleShards - waiting on 2 shards 01:49:50,346 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onShutDown 01:49:50,346 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Initiating leader transfer 01:49:50,346 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Initiating leader transfer 01:49:50,349 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3bedf49 01:49:50,349 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3bedf49 01:49:50,349 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Attempting to transfer leadership 01:49:50,349 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@416f7d6a 01:49:50,349 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@416f7d6a 01:49:50,349 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Attempting to transfer leadership 01:49:50,350 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:49:50,350 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:49:50,350 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:49:50,350 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:49:50,350 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:49:50,350 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:50,350 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:50,351 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:49:50,354 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards] 01:49:50,354 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards] 01:49:50,354 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:49:50,354 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:50,354 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:50,354 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:49:50,354 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:49:50,354 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:49:50,358 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testTransactionChainWithMultipleShards, matchIndex: 4, lastIndex: 4, isVoting: true 01:49:50,358 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:49:50,358 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionChainWithMultipleShards, matchIndex: 3, lastIndex: 3, isVoting: true 01:49:50,358 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:49:50,358 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:49:50,358 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:49:50,358 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Leader transfer complete 01:49:50,358 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Leader transfer complete 01:49:50,764 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All shards shutdown - sending PoisonPill to self 01:49:50,765 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-testTransactionChainWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-1996301231]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:49:50,765 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-1-shard-people-testTransactionChainWithMultipleShards 01:49:50,765 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionChainWithMultipleShards 01:49:50,765 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards (Leader)_slicer-id-415: Closing 01:49:50,765 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader)_slicer-id-414: Closing 01:49:50,765 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 01:49:50,765 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 01:49:50,766 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithMultipleShards 01:49:50,766 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 01:49:50,766 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:49:50,766 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 01:49:50,766 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: shutting down 01:49:50,767 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 01:49:50,768 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:50,768 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:50,777 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:50,778 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:50,778 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:50,778 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:50,780 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:50,798 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithMultipleShards: Closing 01:49:50,800 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionChainWithMultipleShards 01:49:50,800 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Closing 01:49:50,800 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 01:49:50,800 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-people-testTransactionChainWithMultipleShards 01:49:50,800 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Closing 01:49:50,800 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 01:49:50,801 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithMultipleShards 01:49:50,801 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:50,801 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:51,784 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-3991115782882141834] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:51,823 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4361630657696220976] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:51,825 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:51,825 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:49:51.829] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:51,841 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:51,841 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:51,844 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:51.845] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:51,862 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:49:51,874 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2988721631194222315] 01:49:51,874 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:51,876 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:51,876 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:51,877 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:51,877 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:51,889 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:51,889 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:51,889 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:51,893 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:51,898 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [5215343305255885907] 01:49:51,898 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:51,900 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:51,900 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:51,903 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:51,903 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:51,914 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:49:51,921 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [5703160621914120100] 01:49:51,921 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:51,924 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:51,924 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:51,924 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:51,924 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:51,934 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:51,936 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:51,937 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testInstallSnapshot 01:49:51,961 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testInstallSnapshot 01:49:51,967 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: no initial generation override, starting from 0 01:49:51,967 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 01:49:51,967 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1a6ae88c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@57528a71 01:49:51,968 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#737866637] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:51,968 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testInstallSnapshot 01:49:51,968 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Creating local shard: member-1-shard-cars-testInstallSnapshot 01:49:51,968 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:51,968 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:49:51,969 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@536d1443 01:49:51,969 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testInstallSnapshot, 0, 1710164991967, None)) 01:49:51,971 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164991966,0,0)) 01:49:51,971 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@57528a71 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5b14a970 01:49:51,971 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} started 01:49:51,971 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@343bc14f 01:49:51,972 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:51,972 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testInstallSnapshot 01:49:51,972 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:51,972 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:49:51,972 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:49:51,972 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: updated peer member-1-shard-cars-testInstallSnapshot on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot on shard member-1-shard-cars-testInstallSnapshot whose actor address is Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1631827513] 01:49:51,972 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testInstallSnapshot, persistent : true 01:49:51,973 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testInstallSnapshot with journal batch size 1 01:49:51,973 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1631827513] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:51,973 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-notifier#2001695803 created and ready for shard:member-1-shard-cars-testInstallSnapshot 01:49:51,973 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-notifier#2001695803] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:51,973 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/shard-member-1:snapshot-read#1561319119] 01:49:51,973 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/shard-member-1:snapshot-read#1561319119] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:51,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.SnapshotOffer from Actor[akka://cluster-test/system/in-memory-snapshot-store#1954281014] 01:49:51,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: SnapshotOffer called. 01:49:51,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Moving last applied index from -1 to 5 01:49:51,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=1, votedFor=null 01:49:51,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Applying recovered snapshot 01:49:51,974 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-testInstallSnapshot: clearing clients {} 01:49:51,974 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:51,976 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testInstallSnapshot 01:49:51,977 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testInstallSnapshot: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#752827204] with name member-1-shard-cars-testInstallSnapshot-DTCL-publisher 01:49:51,977 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testInstallSnapshot: state snapshot applied in 3.722 ms 01:49:51,978 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Recovery snapshot applied for member-1-shard-cars-testInstallSnapshot in 3.900 ms: snapshotIndex=5, snapshotTerm=1, journal-size=0 01:49:51,980 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#752827204] 01:49:51,980 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testInstallSnapshot: publishChanges: rootPath: / 01:49:51,981 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#752827204] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:51,981 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:51,981 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testInstallSnapshot: Elapsed time for generation of change events for member-1-shard-cars-testInstallSnapshot-DTCL-publisher: 133.4 μs 01:49:51,981 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Recovery completed in 7.077 ms - Switching actor to Follower - last log index = 5, last log term = 1, snapshot index = 5, snapshot term = 1, journal size = 0 01:49:51,981 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Initializing shard [cars] 01:49:51,981 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-testInstallSnapshot is initialized 01:49:51,981 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 01:49:51,981 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testInstallSnapshot is no longer the leader 01:49:51,981 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from null to Follower 01:49:51,981 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , registered listener akka://cluster-test/user/shardmanager-testInstallSnapshot 01:49:51,982 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from null to Follower 01:49:51,983 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-694526545] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:51,984 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-testInstallSnapshot 01:49:51,986 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Recovery complete : shard-manager-testInstallSnapshot 01:49:51,986 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Creating local shard: member-2-shard-cars-testInstallSnapshot 01:49:51,986 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: no initial generation override, starting from 0 01:49:51,987 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} 01:49:51,987 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1d6bc09b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6d4b1e18 01:49:51,987 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@69748769 01:49:51,987 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testInstallSnapshot, 0, 1710164991987, None)) 01:49:51,987 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164991986,0,0)) 01:49:51,987 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@230d733d 01:49:51,987 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6d4b1e18 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@31bf858b 01:49:51,988 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} started 01:49:51,988 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:51,988 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testInstallSnapshot 01:49:51,989 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Shard created : member-2-shard-cars-testInstallSnapshot, persistent : true 01:49:51,989 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot#-1037621727] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:51,989 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-notifier#1284619194 created and ready for shard:member-2-shard-cars-testInstallSnapshot 01:49:51,990 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Starting recovery for member-2-shard-cars-testInstallSnapshot with journal batch size 1 01:49:51,990 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/shard-member-2:snapshot-read#2091865273] 01:49:51,990 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-notifier#1284619194] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:51,990 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/shard-member-2:snapshot-read#2091865273] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:51,990 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:51,990 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:51,990 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 01:49:51,990 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testInstallSnapshot is no longer the leader 01:49:51,991 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testInstallSnapshot , received role change from null to Follower 01:49:51,991 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Initializing shard [cars] 01:49:51,991 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-2-shard-cars-testInstallSnapshot is initialized 01:49:51,991 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testInstallSnapshot , registered listener akka://cluster-test/user/shardmanager-testInstallSnapshot 01:49:51,991 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-2-shard-cars-testInstallSnapshot from null to Follower 01:49:52,38 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#737866637] 01:49:52,38 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: findLocalShard : cars 01:49:52,39 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1631827513] 01:49:52,46 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:49:52,56 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:52,83 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:52,88 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:52,90 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:52,90 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:49:52,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:49:52,90 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-1-shard-cars-testInstallSnapshot set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:49:52,218 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:52,218 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Follower): Received ElectionTimeout - switching to Candidate 01:49:52,218 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testInstallSnapshot] 01:49:52,218 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=2, votedFor=member-1-shard-cars-testInstallSnapshot 01:49:52,219 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 2 01:49:52,219 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Follower) :- Switching from behavior Follower to Candidate, election term: 2 01:49:52,219 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testInstallSnapshot (Follower): Closing 01:49:52,219 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 01:49:52,219 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testInstallSnapshot is no longer the leader 01:49:52,219 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from Follower to Candidate 01:49:52,219 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from Follower to Candidate 01:49:52,518 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 01:49:52,518 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=3, votedFor=member-1-shard-cars-testInstallSnapshot 01:49:52,518 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 3 01:49:52,560 PM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:52,838 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 01:49:52,838 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=4, votedFor=member-1-shard-cars-testInstallSnapshot 01:49:52,838 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 4 01:49:52,889 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:49:52,889 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:52,890 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:52,890 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot 01:49:52,890 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:49:52,890 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:52,890 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:49:52,890 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:49:52,890 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Peer address for peer member-2-shard-cars-testInstallSnapshot set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:49:52,942 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:49:52,948 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:52,948 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot 01:49:52,948 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:49:52,948 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:52,948 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:49:52,948 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:49:53,148 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 01:49:53,148 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 01:49:53,148 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 5 01:49:53,148 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] to peer member-2-shard-cars-testInstallSnapshot 01:49:53,152 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:53,152 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Cluster state: [] 01:49:53,152 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] with higher term appears reachable, updating term. 01:49:53,152 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Term 5 in "RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1]" message is greater than follower's term 0 - updating term 01:49:53,152 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=null 01:49:53,152 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): In requestVote: RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:53,152 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 01:49:53,152 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:49:53,155 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:49:53,155 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:49:53,156 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Election: Leader has following peers: [member-2-shard-cars-testInstallSnapshot] 01:49:53,156 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: false, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 5 01:49:53,156 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:49:53,156 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: true, hasLeader : true 01:49:53,156 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: became leader with frontend state for [] 01:49:53,156 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from Candidate to Leader 01:49:53,156 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testInstallSnapshot, leaderId=member-1-shard-cars-testInstallSnapshot, leaderPayloadVersion=13] 01:49:53,156 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:53,156 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:53,156 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from Candidate to Leader 01:49:53,156 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testInstallSnapshot: All Shards are ready - data store testInstallSnapshot is ready 01:49:53,158 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testInstallSnapshot: Closing 01:49:53,158 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: getLeaderAddress leaderId = member-1-shard-cars-testInstallSnapshot peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:49:53,158 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 01:49:53,158 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#737866637] 01:49:53,158 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testInstallSnapshot Received follower initial sync status for member-2-shard-cars-testInstallSnapshot status sync done false 01:49:53,158 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:49:53,158 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 Queuing TransactionOperation 01:49:53,158 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testInstallSnapshot, leaderId=member-1-shard-cars-testInstallSnapshot, leaderPayloadVersion=13] 01:49:53,158 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1631827513, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@8012816] 01:49:53,158 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:53,158 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:53,158 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: All Shards are ready - data store testInstallSnapshot is ready 01:49:53,158 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1631827513, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@8012816] 01:49:53,158 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testInstallSnapshot-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1631827513)] for shard cars 01:49:53,158 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 01:49:53,158 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@25fe3c0 01:49:53,159 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testInstallSnapshot updated: matchIndex: -1, nextIndex: 0 01:49:53,159 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 0, leaderLastIndex: 5, leaderSnapShotIndex: 5 01:49:53,159 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Initiating install snapshot to follower member-2-shard-cars-testInstallSnapshot: follower nextIndex: 0, leader snapshotIndex: 5, leader lastIndex: 5, leader log size: 0 01:49:53,159 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex 5 and lastAppliedTerm 1 instead. 01:49:53,159 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=5, lastAppliedTerm=1, lastIndex=5, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-testInstallSnapshot 01:49:53,159 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: lastSequenceNumber prior to capture: 4 01:49:53,159 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 01:49:53,163 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: CaptureSnapshotReply received by actor 01:49:53,163 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Persisting of snapshot done: Snapshot [lastIndex=5, lastTerm=1, lastAppliedIndex=5, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@624520b9, electionTerm=5, electionVotedFor=member-1-shard-cars-testInstallSnapshot, ServerConfigPayload=null] 01:49:53,163 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 5 and term: 1 01:49:53,164 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): sendInstallSnapshot 01:49:53,164 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): next snapshot chunk size for follower member-2-shard-cars-testInstallSnapshot: 376 01:49:53,164 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-testInstallSnapshot, member-2-shard-cars-testInstallSnapshot), Chunk: 1/1 01:49:53,164 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: SaveSnapshotSuccess received for snapshot, sequenceNr: 4 01:49:53,164 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Snapshot success - sequence number: 4 01:49:53,164 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: SnapshotComplete received 01:49:53,165 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot Received follower initial sync status for member-2-shard-cars-testInstallSnapshot status sync done true 01:49:53,167 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: handled a snapshot response 01:49:53,167 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: handled a journal response 01:49:53,172 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): handleInstallSnapshot: InstallSnapshot [term=5, leaderId=member-1-shard-cars-testInstallSnapshot, lastIncludedIndex=5, lastIncludedTerm=1, datasize=376, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 01:49:53,174 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 01:49:53,174 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Snapshot installed from leader: member-1-shard-cars-testInstallSnapshot 01:49:53,176 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Applying snapshot on follower: Snapshot [lastIndex=5, lastTerm=1, lastAppliedIndex=5, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5bcf3b13, electionTerm=5, electionVotedFor=member-1-shard-cars-testInstallSnapshot, ServerConfigPayload=null] 01:49:53,176 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - lastSequenceNumber prior to persisting applied snapshot: 2 01:49:53,176 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: SaveSnapshotSuccess received for snapshot, sequenceNr: 2 01:49:53,176 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Snapshot success - sequence number: 2 01:49:53,176 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Moving last applied index from -1 to 5 01:49:53,176 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 01:49:53,176 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Applying snapshot 01:49:53,176 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 01:49:53,176 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-cars-testInstallSnapshot: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 01:49:53,176 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Issuing gracefulStop to shard member-1-shard-cars-testInstallSnapshot 01:49:53,176 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-testInstallSnapshot - waiting on 1 shards 01:49:53,176 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testInstallSnapshot: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-1368831778] with name member-2-shard-cars-testInstallSnapshot-DTCL-publisher 01:49:53,176 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onShutDown 01:49:53,176 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Initiating leader transfer 01:49:53,177 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testInstallSnapshot: state snapshot applied in 369.4 μs 01:49:53,177 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Done applying snapshot 01:49:53,177 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testInstallSnapshot, leaderId=null, leaderPayloadVersion=13] 01:49:53,177 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:53,177 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:53,177 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@51bf2436 01:49:53,177 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testInstallSnapshot: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@51bf2436 01:49:53,177 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Attempting to transfer leadership 01:49:53,177 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-1368831778] 01:49:53,177 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testInstallSnapshot: publishChanges: rootPath: / 01:49:53,177 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-1368831778] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:53,179 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=5, followerId=member-2-shard-cars-testInstallSnapshot, chunkIndex=1, success=true] 01:49:53,179 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: SnapshotComplete received 01:49:53,179 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: handled a journal response 01:49:53,181 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testInstallSnapshot: Elapsed time for generation of change events for member-2-shard-cars-testInstallSnapshot-DTCL-publisher: 105.7 μs 01:49:53,181 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: handled a snapshot response 01:49:53,182 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testInstallSnapshot] 01:49:53,183 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testInstallSnapshot, leaderId=null, leaderPayloadVersion=13] 01:49:53,183 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:53,183 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:53,184 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=5, followerId=member-2-shard-cars-testInstallSnapshot, chunkIndex=1, success=true] 01:49:53,184 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Snapshot successfully installed on follower member-2-shard-cars-testInstallSnapshot (last chunk 1) - matchIndex set to 5, nextIndex set to 6 01:49:53,389 PM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testInstallSnapshot: Failed to transfer leadership in 211.9 ms 01:49:53,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: In unpauseLeader 01:49:53,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: leader transfer failed - sending PoisonPill 01:49:53,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Leadership transfer expired 01:49:53,390 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testInstallSnapshot 01:49:53,390 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testInstallSnapshot (Leader)_slicer-id-420: Closing 01:49:53,390 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Aborting 0 pending queued transactions 01:49:53,392 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: All shards shutdown - sending PoisonPill to self 01:49:53,392 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntriesReply] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot#-1037621727] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1631827513] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1631827513] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:53,392 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-testInstallSnapshot 01:49:53,392 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 01:49:53,392 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 01:49:53,392 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: shutting down 01:49:53,392 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 01:49:53,392 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#-749291162] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-749291162] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:53,393 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 01:49:53,393 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#737866637] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#737866637] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:53,393 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#737866637] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#737866637] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:53,393 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#737866637] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#737866637] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:53,393 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:53,393 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:53,399 PM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:53,399 PM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:53,399 PM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:53,399 PM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:53,401 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:53,409 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testInstallSnapshot: Closing 01:49:53,419 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testInstallSnapshot 01:49:53,419 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testInstallSnapshot (Follower): Closing 01:49:53,419 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Aborting 0 pending queued transactions 01:49:53,419 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testInstallSnapshot 01:49:53,420 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:53,420 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:54,411 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2988721631194222315] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:54,440 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:54,440 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:54,440 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [5215343305255885907] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:54,442 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:54.444] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:54,457 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:54,458 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:54,459 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:49:54.460] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:54,489 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:49:54,499 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [6597279119820016295] 01:49:54,499 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:49:54,501 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:49:54,501 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:49:54,501 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:54,501 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:54,514 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:49:54,514 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:49:54,515 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:49:54,520 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:49:54,530 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-7617653041391512673] 01:49:54,531 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:49:54,532 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:54,532 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:49:54,532 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:54,532 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:54,548 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:49:54,558 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [4966391269448260207] 01:49:54,558 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:49:54,565 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:49:54,565 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:49:54,566 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:49:54,566 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:49:54,573 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:54,583 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:54,585 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 01:49:54,602 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:49:54,609 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-302024130] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,624 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 01:49:54,624 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:54,624 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 01:49:54,624 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:49:54,624 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6cfeadc3 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@60d8b547 01:49:54,624 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5de028a0 01:49:54,624 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:54,624 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:54,624 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1710164994624, None)) 01:49:54,631 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164994623,0,0)) 01:49:54,632 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@60d8b547 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@51ce5c16 01:49:54,632 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3f13336e 01:49:54,640 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:49:54,642 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 01:49:54,642 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:54,642 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:54,642 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:54,668 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:54,668 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:54,668 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:54,668 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1595655369] 01:49:54,669 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 01:49:54,669 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 closeTransaction called 01:49:54,669 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:0_6#745588837)] 01:49:54,669 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 closeTransaction called 01:49:54,669 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:0_7#-1999383556)] 01:49:54,670 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1595655369] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,671 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:54,672 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 01:49:54,678 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 01:49:54,683 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-1:snapshot-read#-1407368679] 01:49:54,683 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-443182374 created and ready for shard:member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:54,683 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:49:54,683 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-443182374] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,683 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-1:snapshot-read#-1407368679] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,690 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:54,690 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:54,690 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Initializing shard [cars] 01:49:54,690 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is initialized 01:49:54,690 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:49:54,690 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 01:49:54,691 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from null to Follower 01:49:54,691 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , registered listener akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader 01:49:54,691 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from null to Follower 01:49:54,692 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:49:54,694 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1159191294] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,695 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:49:54,699 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 01:49:54,699 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:49:54,699 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@416ff5bc to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5b07a19f 01:49:54,700 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5906fe30 01:49:54,700 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1710164994699, None)) 01:49:54,700 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164994698,0,0)) 01:49:54,700 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5b07a19f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6925d206 01:49:54,701 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 01:49:54,706 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 01:49:54,706 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:54,707 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:54,707 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:54,707 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@285bb685 01:49:54,707 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:54,707 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:54,714 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:54,714 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:54,714 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#1077290869] 01:49:54,715 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Shard created : member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 01:49:54,715 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#1077290869] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,715 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-491127194 created and ready for shard:member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:54,715 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 01:49:54,715 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-491127194] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,715 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-2:snapshot-read#1692478753] 01:49:54,716 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-2:snapshot-read#1692478753] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:54,716 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:49:54,716 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:49:54,716 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Initializing shard [cars] 01:49:54,716 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader is initialized 01:49:54,716 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:49:54,716 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 01:49:54,716 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from null to Follower 01:49:54,716 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:54,717 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader , registered listener akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader 01:49:54,717 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader from null to Follower 01:49:54,752 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-302024130] 01:49:54,752 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 01:49:54,752 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1595655369] 01:49:54,990 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:49:54,990 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Received ElectionTimeout - switching to Candidate 01:49:54,990 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] 01:49:54,990 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:54,990 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 1 01:49:54,991 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:49:54,991 PM [cluster-test-shard-dispatcher-14] [DEBUG] MessageAssembler - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Closing 01:49:54,991 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:49:54,991 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 01:49:54,991 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from Follower to Candidate 01:49:54,991 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from Follower to Candidate 01:49:55,300 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 01:49:55,300 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,301 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 2 01:49:55,304 PM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:49:55,522 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:55,522 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,522 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,522 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:55,522 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,522 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,522 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Peer address for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,522 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,523 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:49:55,523 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:49:55,534 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:55,534 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,534 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,534 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:55,534 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,534 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,534 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,582 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:49:55,610 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 01:49:55,610 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,610 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 3 01:49:55,610 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,610 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,613 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:49:55,613 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [] 01:49:55,613 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1595655369] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:55,613 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:49:55,613 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:49:55,613 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=3, votedFor=null 01:49:55,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 01:49:55,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,614 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 01:49:55,617 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 01:49:55,617 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 01:49:55,617 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Election: Leader has following peers: [member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] 01:49:55,617 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:55,617 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:55,617 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 01:49:55,617 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:49:55,618 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: became leader with frontend state for [] 01:49:55,618 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from Candidate to Leader 01:49:55,618 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderPayloadVersion=13] 01:49:55,618 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:55,618 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:55,618 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from Candidate to Leader 01:49:55,618 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 01:49:55,620 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1595655369] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:55,620 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Closing 01:49:55,620 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: getLeaderAddress leaderId = member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,620 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderPayloadVersion=13] 01:49:55,620 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:49:55,620 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:49:55,620 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 01:49:55,620 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader Received follower initial sync status for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader status sync done false 01:49:55,621 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader updated: matchIndex: -1, nextIndex: 0 01:49:55,621 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:49:55,665 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:55,667 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:49:55,671 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 01:49:55,674 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#1153911202] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:55,676 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:49:55,679 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 01:49:55,679 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 01:49:55,679 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:49:55,679 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,679 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2395e5b0 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6f5ac32f 01:49:55,679 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4e405607 01:49:55,679 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1710164995679, None)) 01:49:55,679 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:49:55,679 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,679 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:49:55,679 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,679 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710164995678,0,0)) 01:49:55,679 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:49:55,679 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,680 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4b680d29 01:49:55,680 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6f5ac32f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7d6f9286 01:49:55,681 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 01:49:55,681 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:49:55,681 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,682 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,682 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,682 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#1909820504] 01:49:55,682 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,682 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,682 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#1909820504] 01:49:55,682 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,682 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,682 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#1909820504] 01:49:55,682 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 01:49:55,682 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#1909820504] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:55,683 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-1479390819 created and ready for shard:member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,683 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-1479390819] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:55,683 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-3:snapshot-read#-394723486] 01:49:55,683 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 01:49:55,683 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-3:snapshot-read#-394723486] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:49:55,683 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:49:55,688 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:49:55,688 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 01:49:55,688 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:49:55,688 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:49:55,688 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:49:55,696 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:55,696 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:55,696 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:55,696 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:49:55,699 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:49:55,724 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-302024130] 01:49:55,724 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 01:49:55,725 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1595655369] 01:49:55,725 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1595655369] 01:49:55,751 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader Received follower initial sync status for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader status sync done true 01:49:55,975 PM [cluster-test-shard-dispatcher-28] [WARN] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 218, lastApplied : -1, commitIndex : -1 01:49:56,541 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:49:56,705 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [4966391269448260207] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:56,710 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [4966391269448260207] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:49:59,771 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 01:50:01,645 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 01:50:01,647 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:01,648 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:01,665 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 01:50:01,665 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:01,764 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1159191294] 01:50:01,764 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 01:50:01,764 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#1077290869] 01:50:01,764 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1595655369] 01:50:01,765 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:01,765 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:01,765 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 01:50:01,765 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 01:50:01,765 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=4, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 01:50:01,765 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=4, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 01:50:01,765 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): In requestVote: RequestVote [term=4, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 01:50:01,765 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 01:50:01,765 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): requestVote returning: RequestVoteReply [term=3, voteGranted=false] 01:50:01,765 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=false] 01:50:01,765 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:01,765 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:01,765 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 01:50:01,765 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 01:50:01,765 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=6, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 01:50:01,765 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): In requestVote: RequestVote [term=6, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 01:50:01,765 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=6, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 01:50:01,765 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): requestVote returning: RequestVoteReply [term=3, voteGranted=false] 01:50:01,765 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=6, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 01:50:01,765 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#1077290869] to Actor[akka://cluster-test/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:01,765 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=false] 01:50:01,765 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#1077290869] to Actor[akka://cluster-test/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:01,765 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1595655369] to Actor[akka://cluster-test/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:01,765 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1595655369] to Actor[akka://cluster-test/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:01,765 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1595655369] 01:50:01,766 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:50:01,766 PM [main] [WARN] ActorUtils - ShardManager for testSemiReachableCandidateNotDroppingLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.testSemiReachableCandidateNotDroppingLeader(DistributedDataStoreRemotingIntegrationTest.java:1322) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:50:01,767 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:50:01,767 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:50:01,767 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Issuing gracefulStop to shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:50:01,767 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader - waiting on 1 shards 01:50:01,767 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onShutDown 01:50:01,767 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Initiating leader transfer 01:50:01,768 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@255b8e46 01:50:01,768 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 01:50:01,768 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@255b8e46 01:50:01,768 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Attempting to transfer leadership 01:50:01,768 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:01,768 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:01,768 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:01,768 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:01,770 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader] 01:50:01,770 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 01:50:01,770 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:01,770 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:01,772 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, matchIndex: -1, lastIndex: -1, isVoting: true 01:50:01,772 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Follower's log matches - sending ElectionTimeout 01:50:01,772 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:01,772 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:01,772 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Leader transfer complete 01:50:02,180 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All shards shutdown - sending PoisonPill to self 01:50:02,180 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1595655369]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:02,180 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:50:02,180 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader)_slicer-id-425: Closing 01:50:02,180 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 01:50:02,181 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:50:02,181 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:50:02,181 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:50:02,181 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: shutting down 01:50:02,181 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:50:02,182 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:50:02,182 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-302024130] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-302024130] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:02,182 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-302024130] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-302024130] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:02,182 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-302024130] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-302024130] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:02,182 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#931511077] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#931511077] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:02,184 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:02,184 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:02,645 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Closing 01:50:02,645 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 01:50:03,202 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:03,202 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:50:03.205] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:03,206 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [6597279119820016295] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:03,216 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:50:03,217 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:50:03,217 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Closing 01:50:03,217 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 01:50:03,218 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:50:03,218 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:03,218 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:03,221 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:03.222] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:03,242 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:03,250 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2722575165794462325] 01:50:03,251 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:03,252 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:03,252 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:03,261 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:03,261 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:03,262 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:03,262 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:03,262 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:03,266 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:03,273 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-3479820356122549665] 01:50:03,274 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:03,275 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:03,275 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:03,283 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:03,283 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:03,313 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:03,322 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2253634702150536296] 01:50:03,322 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:03,326 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:03,326 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:03,326 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:03,326 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:03,331 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:03,334 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:03,336 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadyLocalTransactionForwardedToLeader 01:50:03,349 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-857147601] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,350 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 01:50:03,356 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: no initial generation override, starting from 0 01:50:03,356 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 01:50:03,356 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@113c32eb to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@43625fd8 01:50:03,356 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testReadyLocalTransactionForwardedToLeader 01:50:03,356 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Creating local shard: member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:03,356 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:03,356 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:03,360 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@a63a50e 01:50:03,361 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, 0, 1710165003356, None)) 01:50:03,361 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165003355,0,0)) 01:50:03,361 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@b14083b 01:50:03,361 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@43625fd8 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3db0bdec 01:50:03,361 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} started 01:50:03,362 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:03,362 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:03,362 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:03,362 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:03,362 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:03,362 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: updated peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-372681122] 01:50:03,362 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Shard created : member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, persistent : true 01:50:03,363 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:03,363 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with journal batch size 1 01:50:03,368 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadyLocalTransactionForwardedToLeader 01:50:03,370 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#861684545 created and ready for shard:member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:03,370 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-1:snapshot-read#458099873] 01:50:03,374 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:03,374 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:03,374 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Initializing shard [cars] 01:50:03,374 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is initialized 01:50:03,374 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:03,374 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 01:50:03,374 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from null to Follower 01:50:03,374 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader 01:50:03,374 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from null to Follower 01:50:03,378 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-372681122] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,378 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#861684545] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,378 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-1:snapshot-read#458099873] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,389 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 01:50:03,405 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:03,412 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:03,412 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1011453233] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,412 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: no initial generation override, starting from 0 01:50:03,412 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 01:50:03,412 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@79b4896e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1bdbd40 01:50:03,423 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@572b77c6 01:50:03,432 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, 0, 1710165003432, None)) 01:50:03,433 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165003431,0,0)) 01:50:03,433 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1bdbd40 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@155e794f 01:50:03,434 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadyLocalTransactionForwardedToLeader 01:50:03,434 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Creating local shard: member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:03,435 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3d2c5636 01:50:03,440 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:03,441 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:03,442 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} started 01:50:03,442 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:03,442 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:03,451 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:03,451 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:03,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:03,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: updated peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-326068321] 01:50:03,452 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, persistent : true 01:50:03,452 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-326068321] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,452 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#-1187931959 created and ready for shard:member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:03,452 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#-1187931959] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,452 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-2:snapshot-read#1268670210] 01:50:03,452 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-2:snapshot-read#1268670210] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:03,452 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with journal batch size 1 01:50:03,456 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:03,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:03,459 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:03,459 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Initializing shard [cars] 01:50:03,459 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader is initialized 01:50:03,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:03,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 01:50:03,461 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:03,462 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from null to Follower 01:50:03,462 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader 01:50:03,462 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader from null to Follower 01:50:03,493 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-857147601] 01:50:03,493 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 01:50:03,493 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-372681122] 01:50:03,609 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:03,609 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Received ElectionTimeout - switching to Candidate 01:50:03,609 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] 01:50:03,609 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:03,609 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 1 01:50:03,609 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:03,609 PM [cluster-test-shard-dispatcher-13] [DEBUG] MessageAssembler - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Closing 01:50:03,609 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:03,609 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 01:50:03,609 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from Follower to Candidate 01:50:03,609 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from Follower to Candidate 01:50:03,918 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:50:03,918 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:03,918 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 2 01:50:04,33 PM [cluster-test-shard-dispatcher-30] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:04,209 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:50:04,209 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,209 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 3 01:50:04,280 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:04,281 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:04,281 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:04,281 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,281 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,281 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:04,281 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,281 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,281 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,289 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:04,289 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,289 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,289 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:04,289 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,289 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,337 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:04,469 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:50:04,469 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,469 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 4 01:50:04,469 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,472 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:04,473 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Cluster state: [] 01:50:04,473 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:04,473 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:04,473 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=null 01:50:04,473 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:04,473 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,473 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:04,475 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:04,475 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:04,475 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] 01:50:04,475 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:04,475 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:04,475 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:50:04,475 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: became leader with frontend state for [] 01:50:04,475 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from Candidate to Leader 01:50:04,475 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderPayloadVersion=13] 01:50:04,475 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:04,475 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:04,475 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from Candidate to Leader 01:50:04,475 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All Shards are ready - data store testReadyLocalTransactionForwardedToLeader is ready 01:50:04,476 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Closing 01:50:04,476 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,476 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader status sync done false 01:50:04,476 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: -1, nextIndex: 0 01:50:04,476 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderPayloadVersion=13] 01:50:04,476 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:04,476 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:04,476 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:04,477 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All Shards are ready - data store testReadyLocalTransactionForwardedToLeader is ready 01:50:04,552 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1011453233] 01:50:04,552 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 01:50:04,553 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-326068321] 01:50:04,553 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,553 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1011453233] 01:50:04,553 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 01:50:04,553 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-326068321] 01:50:04,565 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: handleReadyLocalTransaction for member-1-ShardTransactionTest-fe-0-txn-197-0 01:50:04,565 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,565 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Forwarding ReadyLocalTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader)] 01:50:04,567 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, disabling transaction tracking 01:50:04,567 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:50:04,567 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:50:04,568 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:04,568 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:04,568 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,568 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 01:50:04,568 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-197-0 01:50:04,568 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-197-0 of 1 operations, client version 13 01:50:04,568 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-197-0 01:50:04,568 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-197-0 ready 01:50:04,568 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-197-0 01:50:04,568 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 validated 01:50:04,568 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:04,568 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-197-0: success 01:50:04,568 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-197-0 01:50:04,568 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-197-0 01:50:04,568 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 candidate ready 01:50:04,568 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-197-0: canCommit - candidate rootPath: / 01:50:04,569 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-197-0: canCommit - messages: [] 01:50:04,569 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-197-0: preCommit - successfulFromPrevious: [] 01:50:04,569 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 prepared 01:50:04,569 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:04,569 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-197-0 01:50:04,569 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-197-0 01:50:04,569 PM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 197 01:50:04,569 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:50:04,569 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:50:04,569 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-197-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:04,569 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:04,569 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,569 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 submitted to persistence 01:50:04,570 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:50:04,570 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:50:04,570 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 01:50:04,570 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:50:04,570 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:04,570 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:50:04,570 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:50:04,571 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 1 01:50:04,571 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:04,572 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:50:04,572 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader status sync done true 01:50:04,572 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 0, nextIndex: 1 01:50:04,572 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:04,572 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 0 01:50:04,572 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from -1 to 0 01:50:04,572 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:50:04,572 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 01:50:04,572 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 01:50:04,572 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, nanosAgo=5088010} is already disabled 01:50:04,573 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:04,573 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,573 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 01:50:04,573 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 01:50:04,574 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 01:50:04,574 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:50:04,574 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:04,574 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log Simple{size=197} 01:50:04,574 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:50:04,574 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 2 01:50:04,574 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 0 01:50:04,574 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:04,574 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:50:04,574 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 0 01:50:04,574 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from -1 to 0 01:50:04,574 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:50:04,574 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 01:50:04,574 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 01:50:04,574 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:50:04,575 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 01:50:04,575 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 1, nextIndex: 2 01:50:04,575 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:04,575 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 1 01:50:04,575 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 0 to 1 01:50:04,575 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 01:50:04,575 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-197-0 01:50:04,575 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-197-0: commit - successfulFromPrevious: [] 01:50:04,575 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 changing state from COMMIT_PENDING to COMMITTED 01:50:04,575 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-197-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/system/testProbe-761#648125667] 01:50:04,576 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#41960287] with name member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher 01:50:04,576 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#41960287] 01:50:04,576 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 01:50:04,576 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 01:50:04,576 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:04,576 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 137.2 μs 01:50:04,576 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 01:50:04,576 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#41960287] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:04,577 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 1 01:50:04,577 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:04,577 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 1 01:50:04,577 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 0 to 1 01:50:04,577 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 01:50:04,578 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-197-0 01:50:04,578 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-1331047199] with name member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher 01:50:04,578 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 01:50:04,578 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 01:50:04,578 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 01:50:04,578 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-857147601] 01:50:04,578 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-1331047199] 01:50:04,578 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 Queuing TransactionOperation 01:50:04,578 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 01:50:04,578 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-1331047199] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:04,579 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 71.95 μs 01:50:04,579 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:04,579 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-372681122, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2a387854] 01:50:04,579 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-372681122, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2a387854] 01:50:04,579 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-372681122)] for shard cars 01:50:04,579 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 01:50:04,579 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4f12b4b5 01:50:04,580 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: handleReadyLocalTransaction for member-1-ShardTransactionTest-fe-0-txn-198-0 01:50:04,580 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,580 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Forwarding ReadyLocalTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader)] 01:50:04,581 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-198-0 01:50:04,581 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-198-0 of 1 operations, client version 13 01:50:04,581 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-198-0 01:50:04,582 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-198-0 ready 01:50:04,584 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0 canCommit 01:50:04,584 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-372681122)] resolved 01:50:04,584 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: successfully resolved all cohort actors 01:50:04,584 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0 finishCanCommit 01:50:04,584 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: sending CanCommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-198-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-372681122)] 01:50:04,584 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-372681122)] 01:50:04,584 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Can committing transaction member-1-ShardTransactionTest-fe-0-txn-198-0 01:50:04,584 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-198-0 01:50:04,584 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 validated 01:50:04,584 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:04,584 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-198-0: success 01:50:04,585 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:50:04,585 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: canCommit returning result: true 01:50:04,585 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0 commit 01:50:04,585 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0 finish commit 01:50:04,585 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: Sending CommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-198-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-372681122)] 01:50:04,585 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-372681122)] 01:50:04,585 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-198-0 01:50:04,585 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-198-0 01:50:04,585 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 candidate ready 01:50:04,585 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-198-0: canCommit - candidate rootPath: / 01:50:04,585 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-198-0: canCommit - messages: [] 01:50:04,585 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-198-0: preCommit - successfulFromPrevious: [] 01:50:04,585 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 prepared 01:50:04,585 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:04,585 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-198-0 01:50:04,585 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-198-0 01:50:04,585 PM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 208 01:50:04,585 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:50:04,585 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:50:04,585 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-198-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:04,585 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:50:04,585 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,585 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 01:50:04,585 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 submitted to persistence 01:50:04,586 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:50:04,587 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 01:50:04,587 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:50:04,587 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:50:04,587 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log Simple{size=208} 01:50:04,587 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:50:04,587 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 2 01:50:04,587 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:04,587 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:50:04,588 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 2, nextIndex: 3 01:50:04,588 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:50:04,588 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 2 01:50:04,588 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 1 to 2 01:50:04,588 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 01:50:04,588 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-198-0 01:50:04,588 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-198-0: commit - successfulFromPrevious: [] 01:50:04,588 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 changing state from COMMIT_PENDING to COMMITTED 01:50:04,589 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-198-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testReadyLocalTransactionForwardedToLeader_member-1-shard-cars-testReadyLocalTransactionForwardedToLeader%23-372681122$a] 01:50:04,589 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-198-0: commit succeeded 01:50:04,589 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0 01:50:04,589 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 01:50:04,589 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-372681122)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:04,589 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4f12b4b5 01:50:04,589 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:04,589 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 01:50:04,589 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:04,589 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 01:50:04,589 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 01:50:04,589 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 118.6 μs 01:50:04,589 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Issuing gracefulStop to shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:04,589 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader - waiting on 1 shards 01:50:04,590 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onShutDown 01:50:04,590 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Initiating leader transfer 01:50:04,590 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1a06d4a3 01:50:04,590 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1a06d4a3 01:50:04,590 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Attempting to transfer leadership 01:50:04,590 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:04,590 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 01:50:04,590 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:04,590 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:04,592 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 2 01:50:04,592 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:50:04,592 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 2 01:50:04,592 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 1 to 2 01:50:04,592 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 01:50:04,592 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-198-0 01:50:04,592 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 01:50:04,592 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 01:50:04,593 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 01:50:04,593 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 107.5 μs 01:50:04,593 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, matchIndex: 2, lastIndex: 2, isVoting: true 01:50:04,593 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Follower's log matches - sending ElectionTimeout 01:50:04,593 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:04,593 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Leader transfer complete 01:50:04,594 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader] 01:50:04,594 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 01:50:04,594 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:04,594 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:05,9 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 01:50:05,9 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-372681122]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:05,9 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:05,9 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader)_slicer-id-432: Closing 01:50:05,10 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Aborting 0 pending queued transactions 01:50:05,10 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 01:50:05,10 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 01:50:05,10 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 01:50:05,10 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 01:50:05,10 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: shutting down 01:50:05,10 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 01:50:05,11 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-857147601] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-857147601] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:05,11 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-857147601] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-857147601] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:05,11 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-857147601] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-857147601] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:05,11 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-857147601] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-857147601] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:05,12 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:05,12 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:05,22 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:05,22 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:05,22 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:05,22 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:05,28 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:05,43 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 01:50:05,44 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:50:05,44 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Closing 01:50:05,44 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Aborting 0 pending queued transactions 01:50:05,44 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 01:50:05,47 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:05,47 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:06,26 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2722575165794462325] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:06,65 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:06,65 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:06,66 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-3479820356122549665] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:06,69 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:06.071] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:06,84 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:06,84 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/11/2024 13:50:06.091] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:06,105 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:06,117 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [5373088571363701158] 01:50:06,118 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:06,119 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:06,119 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:06,120 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:06,120 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:06,131 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:06,131 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:06,131 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:06,134 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:06,141 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [3420056879562588265] 01:50:06,141 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:06,141 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:06,141 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:06,157 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:06,159 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:06,159 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:06,165 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [776568828508741197] 01:50:06,165 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:06,172 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:06,172 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:06,175 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:06,176 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:06,182 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithSingleShard 01:50:06,203 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithSingleShard 01:50:06,205 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-904215119] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:06,237 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithSingleShard 01:50:06,237 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Creating local shard: member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:50:06,237 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:06,237 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:50:06,240 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: no initial generation override, starting from 0 01:50:06,240 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:50:06,240 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@13f3469d to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@eb93ffd 01:50:06,240 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@51b9c297 01:50:06,241 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, 0, 1710165006240, None)) 01:50:06,241 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165006239,0,0)) 01:50:06,241 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6fa41548 01:50:06,241 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@eb93ffd to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@63c552ca 01:50:06,241 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} started 01:50:06,242 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:06,244 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:06,246 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithSingleShard 01:50:06,247 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:06,247 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:50:06,248 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:06,248 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:06,264 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithSingleShard 01:50:06,264 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#1571053822] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:06,280 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: no initial generation override, starting from 0 01:50:06,280 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:50:06,280 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@69769c90 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1192140a 01:50:06,280 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5490985c 01:50:06,280 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, 0, 1710165006280, None)) 01:50:06,280 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165006279,0,0)) 01:50:06,281 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1192140a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@47b49412 01:50:06,281 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} started 01:50:06,282 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithSingleShard 01:50:06,283 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:50:06,283 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1c462c7c 01:50:06,283 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:06,283 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:50:06,284 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testReadWriteTransactionWithSingleShard, persistent : true 01:50:06,285 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadWriteTransactionWithSingleShard with journal batch size 1 01:50:06,285 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard#1094510809] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:06,285 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-2:snapshot-read#187254518] 01:50:06,285 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-notifier#914318776 created and ready for shard:member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:50:06,285 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-notifier#914318776] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:06,285 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-2:snapshot-read#187254518] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:06,286 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:06,286 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:06,286 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Initializing shard [cars] 01:50:06,286 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard is initialized 01:50:06,286 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:06,286 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 01:50:06,286 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithSingleShard , received role change from null to Follower 01:50:06,286 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard 01:50:06,286 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testReadWriteTransactionWithSingleShard from null to Follower 01:50:06,289 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:50:06,289 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:50:06,289 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testReadWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard on shard member-1-shard-cars-testReadWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-752152960] 01:50:06,289 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-cars-testReadWriteTransactionWithSingleShard, persistent : true 01:50:06,289 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadWriteTransactionWithSingleShard with journal batch size 1 01:50:06,289 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-752152960] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:06,289 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-notifier#878920544 created and ready for shard:member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:50:06,290 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-notifier#878920544] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:06,290 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-1:snapshot-read#1620821732] 01:50:06,290 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-1:snapshot-read#1620821732] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:06,290 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:06,290 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:06,290 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:06,290 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 01:50:06,290 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Initializing shard [cars] 01:50:06,290 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is initialized 01:50:06,290 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from null to Follower 01:50:06,290 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard 01:50:06,290 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from null to Follower 01:50:06,301 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:06,303 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:06,306 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:06,308 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:06,308 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:50:06,308 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:50:06,308 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:50:06,308 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:06,332 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-904215119] 01:50:06,332 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: findLocalShard : cars 01:50:06,332 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-752152960] 01:50:06,521 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:06,521 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 01:50:06,521 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadWriteTransactionWithSingleShard] 01:50:06,521 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:50:06,521 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 1 01:50:06,521 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:06,521 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Closing 01:50:06,521 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:06,521 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 01:50:06,522 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from Follower to Candidate 01:50:06,522 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from Follower to Candidate 01:50:06,751 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:50:06,751 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:50:06,751 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 2 01:50:06,892 PM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:07,21 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:50:07,21 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:50:07,21 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 3 01:50:07,133 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:07,133 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:07,133 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard 01:50:07,133 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:07,133 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:50:07,133 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:07,133 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:50:07,133 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:50:07,134 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:50:07,192 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:07,192 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard 01:50:07,192 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:50:07,192 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:07,192 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:50:07,192 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:50:07,192 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:07,321 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:50:07,321 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:50:07,321 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 4 01:50:07,321 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:50:07,331 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:07,331 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Cluster state: [] 01:50:07,331 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:07,331 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:07,331 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 01:50:07,331 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:07,331 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:50:07,331 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:07,332 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:07,332 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:07,332 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadWriteTransactionWithSingleShard] 01:50:07,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:07,333 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:07,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:50:07,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: became leader with frontend state for [] 01:50:07,333 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from Candidate to Leader 01:50:07,333 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:50:07,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:07,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:07,333 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from Candidate to Leader 01:50:07,333 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All Shards are ready - data store testReadWriteTransactionWithSingleShard is ready 01:50:07,334 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Closing 01:50:07,334 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testReadWriteTransactionWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:50:07,334 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithSingleShard status sync done false 01:50:07,334 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: -1, nextIndex: 0 01:50:07,334 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:07,338 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:50:07,338 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:07,338 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:07,338 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All Shards are ready - data store testReadWriteTransactionWithSingleShard is ready 01:50:07,343 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testReadWriteTransactionWithSingleShard commit rateLimit adjusted to 131.69507394001653 01:50:07,343 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:07,343 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#1571053822] 01:50:07,343 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:50:07,343 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:50:07,343 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:50:07,343 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:50:07,344 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:07,344 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard, primaryVersion=13] 01:50:07,346 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard, primaryVersion=13] 01:50:07,346 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] for shard cars 01:50:07,346 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] found - trying create transaction 01:50:07,346 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] 01:50:07,347 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, disabling transaction tracking 01:50:07,348 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 01:50:07,348 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 01:50:07,348 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:07,348 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:07,348 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:50:07,348 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}]]] 01:50:07,348 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:07,348 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-1102351300] 01:50:07,349 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-1102351300] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:07,349 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 01:50:07,350 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}]]] 01:50:07,350 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 01:50:07,350 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:07,350 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66} 01:50:07,350 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-1102351300, transactionId=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0, version=13] 01:50:07,350 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 01:50:07,350 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:07,350 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Log size is now 1 01:50:07,350 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:50:07,350 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:07,350 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 01:50:07,351 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:50:07,351 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 sending 3 batched modifications, ready: false 01:50:07,351 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-1102351300)] 01:50:07,351 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-1102351300)] 01:50:07,351 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 01:50:07,351 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithSingleShard status sync done true 01:50:07,353 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: 0, nextIndex: 1 01:50:07,353 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:07,353 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Setting last applied to 0 01:50:07,353 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 01:50:07,353 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66} 01:50:07,353 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:50:07,353 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadWriteTransactionWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} does not match any client, pre-disabling client. 01:50:07,354 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, nanosAgo=5987913} is already disabled 01:50:07,354 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:07,354 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 01:50:07,355 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 modifications completed with BatchedModificationsReply [numBatched=3] 01:50:07,355 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 ReadData operation succeeded 01:50:07,355 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Commit index set to 0 01:50:07,355 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:07,356 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Setting last applied to 0 01:50:07,356 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 01:50:07,356 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66} 01:50:07,356 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:50:07,356 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadWriteTransactionWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} does not match any client, pre-disabling client. 01:50:07,356 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 01:50:07,356 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 01:50:07,356 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeRead DataExists called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 01:50:07,356 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 sending 1 batched modifications, ready: false 01:50:07,356 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-1102351300)] 01:50:07,356 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.DataExists to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-1102351300)] 01:50:07,361 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 modifications completed with BatchedModificationsReply [numBatched=1] 01:50:07,361 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 DataExists operation succeeded 01:50:07,361 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Readying 1 components for commit 01:50:07,361 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Readying transaction for shard cars 01:50:07,362 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 directCommit called 01:50:07,362 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 sending 0 batched modifications, ready: true 01:50:07,362 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#-1102351300)] 01:50:07,362 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 canCommit 01:50:07,363 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:07,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: handleForwardedReadyTransaction for member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:07,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Readying transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0, client version 13 01:50:07,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: readying transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:07,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 ready 01:50:07,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:07,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 validated 01:50:07,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:07,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: canCommit for member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: success 01:50:07,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Committing transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:07,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:07,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 candidate ready 01:50:07,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:50:07,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: canCommit - messages: [] 01:50:07,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:50:07,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 prepared 01:50:07,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:07,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Finishing commit for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:07,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:07,364 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 249 01:50:07,364 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 01:50:07,364 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 01:50:07,364 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:07,364 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:07,364 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:50:07,364 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}]]] 01:50:07,364 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 submitted to persistence 01:50:07,365 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 01:50:07,365 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}]]] 01:50:07,365 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 01:50:07,365 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:07,365 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=249} 01:50:07,365 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 01:50:07,365 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Log size is now 2 01:50:07,365 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:07,365 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 01:50:07,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: 1, nextIndex: 2 01:50:07,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:07,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Setting last applied to 1 01:50:07,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 01:50:07,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 1 data Simple{size=249} 01:50:07,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:07,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:50:07,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:50:07,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testReadWriteTransactionWithSingleShard_member-1-shard-cars-testReadWriteTransactionWithSingleShard_shard-cars-member-2%3Adatastore-testReadWriteTransactionWithSingleShard%400%3A0_8%23-1102351300$f] 01:50:07,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1516611884] with name member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher 01:50:07,366 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1516611884] 01:50:07,366 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:50:07,366 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1516611884] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:07,366 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher: 110.9 μs 01:50:07,367 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 01:50:07,367 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:07,367 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 01:50:07,368 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 successfully completed direct commit 01:50:07,368 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 modifications completed with CommitTransactionReply [version=13] 01:50:07,368 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 01:50:07,368 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Commit index set to 1 01:50:07,368 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:07,368 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Setting last applied to 1 01:50:07,368 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 01:50:07,368 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 1 data Simple{size=249} 01:50:07,368 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:07,368 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:07,368 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] found - trying create transaction 01:50:07,368 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] 01:50:07,368 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 Queuing TransactionOperation 01:50:07,368 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:07,369 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#1295765352] with name member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher 01:50:07,369 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#1295765352] 01:50:07,369 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 01:50:07,369 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 01:50:07,369 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:50:07,369 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher: 76.47 μs 01:50:07,369 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#1295765352] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:07,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 01:50:07,370 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_9#-812413072] 01:50:07,370 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_9#-812413072] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:07,371 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_9#-812413072, transactionId=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0, version=13] 01:50:07,371 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:50:07,371 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_9#-812413072)] 01:50:07,374 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 ReadData operation succeeded 01:50:07,374 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:50:07,374 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:50:07,374 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithSingleShard - waiting on 1 shards 01:50:07,374 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onShutDown 01:50:07,374 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Initiating leader transfer 01:50:07,374 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:50:07,374 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3fb7f426 01:50:07,374 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:07,374 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3fb7f426 01:50:07,374 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:07,374 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Attempting to transfer leadership 01:50:07,375 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:07,376 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard] 01:50:07,376 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:50:07,376 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:07,376 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:07,376 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadWriteTransactionWithSingleShard, matchIndex: 1, lastIndex: 1, isVoting: true 01:50:07,376 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 01:50:07,376 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:07,376 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Leader transfer complete 01:50:07,791 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 01:50:07,791 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-752152960]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:07,791 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:50:07,791 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader)_slicer-id-437: Closing 01:50:07,791 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:50:07,792 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithSingleShard 01:50:07,792 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 01:50:07,792 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:50:07,792 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: shutting down 01:50:07,792 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 01:50:07,792 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 01:50:07,792 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-904215119] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-904215119] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:07,792 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-904215119] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-904215119] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:07,793 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#-1047739330] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1047739330] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:07,793 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:07,793 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:07,798 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:07,799 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:07,799 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:07,799 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:07,813 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 01:50:07,814 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:50:07,814 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Closing 01:50:07,814 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:50:07,814 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithSingleShard 01:50:07,815 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:07,815 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:08,812 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [5373088571363701158] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:08,831 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [3420056879562588265] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:08,836 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:08,836 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:08,838 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:08.839] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:08,855 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:08,855 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:08,857 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:08.859] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:08,883 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:08,890 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4924890193741761073] 01:50:08,891 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:08,892 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:08,892 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:08,893 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:08,893 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:08,909 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:08,909 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:08,909 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:08,915 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:08,931 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-4544008715493771672] 01:50:08,931 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:08,933 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:08,933 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:08,943 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:08,943 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:08,955 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:08,968 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [146669837198591345] 01:50:08,968 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:08,973 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:08,974 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:08,974 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:08,975 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:08,975 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:08,981 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:08,982 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:08,983 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSnapshotOnRootOverwrite 01:50:08,985 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:08,993 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1533125287] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:08,995 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSnapshotOnRootOverwrite 01:50:09,0 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSnapshotOnRootOverwrite 01:50:09,0 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-1-shard-cars-testSnapshotOnRootOverwrite 01:50:09,1 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-1-shard-default-testSnapshotOnRootOverwrite 01:50:09,1 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:09,1 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:50:09,1 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:50:09,4 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:09,6 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: no initial generation override, starting from 0 01:50:09,6 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:50:09,6 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@328f6354 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@53d146ee 01:50:09,6 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5563bf0a 01:50:09,8 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSnapshotOnRootOverwrite, 0, 1710165009008, None)) 01:50:09,11 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:09,11 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165009007,0,0)) 01:50:09,11 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@53d146ee to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2760bf6e 01:50:09,11 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} started 01:50:09,11 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:09,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@a6082ce 01:50:09,13 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:09,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:09,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSnapshotOnRootOverwrite 01:50:09,13 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:50:09,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:50:09,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-cars-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite on shard member-1-shard-cars-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510] 01:50:09,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-default-testSnapshotOnRootOverwrite 01:50:09,14 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:50:09,14 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:50:09,14 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-default-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite on shard member-1-shard-default-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105] 01:50:09,14 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testSnapshotOnRootOverwrite, persistent : true 01:50:09,14 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSnapshotOnRootOverwrite with journal batch size 1 01:50:09,15 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-default-testSnapshotOnRootOverwrite, persistent : true 01:50:09,15 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSnapshotOnRootOverwrite 01:50:09,15 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-default-testSnapshotOnRootOverwrite with journal batch size 1 01:50:09,19 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSnapshotOnRootOverwrite 01:50:09,20 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#367909887] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,21 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSnapshotOnRootOverwrite 01:50:09,21 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: no initial generation override, starting from 0 01:50:09,21 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:09,21 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:50:09,21 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1c0c85e4 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3a8cf1f 01:50:09,21 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-2-shard-default-testSnapshotOnRootOverwrite 01:50:09,21 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3fa892b 01:50:09,22 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:09,22 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:50:09,22 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:50:09,22 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSnapshotOnRootOverwrite, 0, 1710165009022, None)) 01:50:09,22 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165009021,0,0)) 01:50:09,22 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3a8cf1f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@115f349f 01:50:09,22 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1c70629c 01:50:09,24 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,24 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,24 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-notifier#-446182944 created and ready for shard:member-1-shard-cars-testSnapshotOnRootOverwrite 01:50:09,24 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#1038402747] 01:50:09,24 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-notifier#2082500423 created and ready for shard:member-1-shard-default-testSnapshotOnRootOverwrite 01:50:09,24 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#57461859] 01:50:09,24 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-notifier#-446182944] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,27 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#1038402747] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,27 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-notifier#2082500423] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,27 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#57461859] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,32 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} started 01:50:09,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:09,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:09,33 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:50:09,33 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:09,33 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-cars-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite on shard member-2-shard-cars-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#534475122] 01:50:09,33 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-default-testSnapshotOnRootOverwrite 01:50:09,33 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:50:09,33 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:50:09,33 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-default-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite on shard member-2-shard-default-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite#584159954] 01:50:09,33 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-default-testSnapshotOnRootOverwrite, persistent : true 01:50:09,34 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testSnapshotOnRootOverwrite, persistent : true 01:50:09,34 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-default-testSnapshotOnRootOverwrite with journal batch size 1 01:50:09,34 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSnapshotOnRootOverwrite with journal batch size 1 01:50:09,36 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-notifier#-636823181 created and ready for shard:member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:09,36 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#1291999984] 01:50:09,36 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-notifier#605711081 created and ready for shard:member-2-shard-default-testSnapshotOnRootOverwrite 01:50:09,36 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#1185357987] 01:50:09,36 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite#584159954] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:09,38 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:09,38 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#534475122] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,38 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-notifier#-636823181] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,38 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#1291999984] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,38 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-notifier#605711081] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,38 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:09,38 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:09,38 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#1185357987] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:09,39 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:50:09,39 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 01:50:09,39 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:50:09,39 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [default] 01:50:09,39 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-default-testSnapshotOnRootOverwrite is initialized 01:50:09,39 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [cars] 01:50:09,39 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSnapshotOnRootOverwrite is initialized 01:50:09,39 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSnapshotOnRootOverwrite , received role change from null to Follower 01:50:09,42 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 01:50:09,45 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:50:09,45 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-2-shard-default-testSnapshotOnRootOverwrite is no longer the leader 01:50:09,45 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-default-testSnapshotOnRootOverwrite , received role change from null to Follower 01:50:09,45 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-default-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:50:09,45 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-default-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 01:50:09,46 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-2-shard-cars-testSnapshotOnRootOverwrite from null to Follower 01:50:09,46 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-2-shard-default-testSnapshotOnRootOverwrite from null to Follower 01:50:09,46 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:09,46 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:09,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:50:09,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 01:50:09,47 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from null to Follower 01:50:09,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [cars] 01:50:09,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSnapshotOnRootOverwrite is initialized 01:50:09,47 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:09,47 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:09,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [default] 01:50:09,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-default-testSnapshotOnRootOverwrite is initialized 01:50:09,47 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 01:50:09,47 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from null to Follower 01:50:09,48 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:50:09,48 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-default-testSnapshotOnRootOverwrite is no longer the leader 01:50:09,48 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from null to Follower 01:50:09,48 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 01:50:09,49 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from null to Follower 01:50:09,82 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1533125287] 01:50:09,83 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 01:50:09,83 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510] 01:50:09,329 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:09,329 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Received ElectionTimeout - switching to Candidate 01:50:09,329 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Election: Candidate has following voting peers: [member-2-shard-default-testSnapshotOnRootOverwrite] 01:50:09,329 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=1, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 01:50:09,329 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 1 01:50:09,329 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:09,329 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Closing 01:50:09,329 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:50:09,329 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-default-testSnapshotOnRootOverwrite is no longer the leader 01:50:09,329 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from Follower to Candidate 01:50:09,329 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from Follower to Candidate 01:50:09,348 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:09,348 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Received ElectionTimeout - switching to Candidate 01:50:09,348 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSnapshotOnRootOverwrite] 01:50:09,348 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=1, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 01:50:09,348 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 1 01:50:09,348 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:09,348 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Closing 01:50:09,348 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:50:09,348 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 01:50:09,348 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from Follower to Candidate 01:50:09,348 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from Follower to Candidate 01:50:09,571 PM [cluster-test-shard-dispatcher-31] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:09,611 PM [cluster-test-shard-dispatcher-31] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:09,638 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:50:09,638 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=2, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 01:50:09,638 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 2 01:50:09,668 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:50:09,668 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=2, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 01:50:09,668 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 2 01:50:09,879 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:50:09,879 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 01:50:09,879 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 3 01:50:09,909 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:09,909 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite 01:50:09,909 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:50:09,909 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite 01:50:09,909 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:50:09,909 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:09,909 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:09,909 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:50:09,909 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:50:09,909 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:50:09,909 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:09,910 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:09,910 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:09,910 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-2-shard-default-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:50:09,924 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:09,924 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite 01:50:09,924 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:09,924 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite 01:50:09,924 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:50:09,924 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:09,924 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:09,924 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:09,924 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:50:09,924 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:50:09,938 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:50:09,938 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 01:50:09,938 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 3 01:50:09,938 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:09,941 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:09,941 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Cluster state: [] 01:50:09,941 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:09,941 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:09,941 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=null 01:50:09,941 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:09,941 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 01:50:09,941 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 01:50:09,943 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 01:50:09,943 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 01:50:09,943 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Election: Leader has following peers: [member-2-shard-cars-testSnapshotOnRootOverwrite] 01:50:09,943 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:09,943 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 01:50:09,943 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: true, hasLeader : true 01:50:09,943 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: became leader with frontend state for [] 01:50:09,943 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from Candidate to Leader 01:50:09,943 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 01:50:09,943 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:09,943 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:09,943 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from Candidate to Leader 01:50:09,946 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testSnapshotOnRootOverwrite: Closing 01:50:09,946 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: getLeaderAddress leaderId = member-1-shard-cars-testSnapshotOnRootOverwrite peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:50:09,946 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-cars-testSnapshotOnRootOverwrite status sync done false 01:50:09,946 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSnapshotOnRootOverwrite, leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 01:50:09,946 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:09,946 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:09,947 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: -1, nextIndex: 0 01:50:09,947 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:09,986 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:10,40 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1533125287] 01:50:10,40 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : default 01:50:10,41 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105] 01:50:10,60 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-cars-testSnapshotOnRootOverwrite status sync done true 01:50:10,108 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:50:10,108 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 01:50:10,108 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 4 01:50:10,108 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-default-testSnapshotOnRootOverwrite 01:50:10,110 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:10,110 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Cluster state: [] 01:50:10,110 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:10,110 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:10,110 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=null 01:50:10,110 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:10,110 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 01:50:10,110 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:10,112 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:10,112 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:10,112 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Election: Leader has following peers: [member-2-shard-default-testSnapshotOnRootOverwrite] 01:50:10,112 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:10,112 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:10,112 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: true, hasLeader : true 01:50:10,113 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: became leader with frontend state for [] 01:50:10,113 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from Candidate to Leader 01:50:10,113 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 01:50:10,113 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 01:50:10,113 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from Candidate to Leader 01:50:10,113 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All Shards are ready - data store testSnapshotOnRootOverwrite is ready 01:50:10,113 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 01:50:10,114 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-default-testSnapshotOnRootOverwrite: Closing 01:50:10,114 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: getLeaderAddress leaderId = member-1-shard-default-testSnapshotOnRootOverwrite peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:50:10,114 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-testSnapshotOnRootOverwrite, leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 01:50:10,114 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 01:50:10,114 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 01:50:10,114 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All Shards are ready - data store testSnapshotOnRootOverwrite is ready 01:50:10,114 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-default-testSnapshotOnRootOverwrite status sync done false 01:50:10,115 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: -1, nextIndex: 0 01:50:10,115 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:10,142 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testSnapshotOnRootOverwrite commit rateLimit adjusted to 162.12298098119095 01:50:10,142 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,143 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1533125287] 01:50:10,144 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1533125287] 01:50:10,144 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:10,144 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@1e345ba] 01:50:10,144 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:50:10,144 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@409b4b1c] 01:50:10,144 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@1e345ba] 01:50:10,145 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars 01:50:10,145 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 01:50:10,145 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,145 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@409b4b1c] 01:50:10,145 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105)] for shard default 01:50:10,145 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Shard default resolved to local data tree - adding local factory 01:50:10,145 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 - Creating local component for shard default using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@131c21b0 01:50:10,145 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 Readying 2 components for commit 01:50:10,145 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 Readying transaction for shard cars 01:50:10,145 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 = [ 1 ] 01:50:10,145 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,146 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 Readying transaction for shard default 01:50:10,146 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 = [ 1 ] 01:50:10,146 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105)] 01:50:10,146 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 canCommit 01:50:10,146 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,146 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, disabling transaction tracking 01:50:10,146 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 01:50:10,146 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 01:50:10,146 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:10,146 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,146 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, disabling transaction tracking 01:50:10,146 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:10,146 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 01:50:10,146 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 01:50:10,146 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,146 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:10,146 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}]]] 01:50:10,146 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:10,146 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 01:50:10,146 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,146 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}]]] 01:50:10,146 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,147 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionReadyReplyMapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 readyTransaction 01:50:10,147 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105)] resolved 01:50:10,147 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 resolved to actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105)] 01:50:10,147 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionReadyReplyMapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 readyTransaction 01:50:10,147 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] resolved 01:50:10,147 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: successfully resolved all cohort actors 01:50:10,147 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 finishCanCommit 01:50:10,147 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,147 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,147 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 resolved to actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,147 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Can committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,147 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,147 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 validated 01:50:10,147 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:10,147 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: success 01:50:10,147 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:50:10,147 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105)] 01:50:10,148 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105)] 01:50:10,148 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Can committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,148 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,148 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 validated 01:50:10,148 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:10,148 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: success 01:50:10,148 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:50:10,148 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit returning result: true 01:50:10,148 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 commit 01:50:10,148 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 finish commit 01:50:10,148 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,148 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,148 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105)] 01:50:10,148 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105)] 01:50:10,148 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 01:50:10,148 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,148 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 01:50:10,148 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,148 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 candidate ready 01:50:10,148 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:50:10,148 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,148 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,148 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 candidate ready 01:50:10,148 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:50:10,148 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - messages: [] 01:50:10,148 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - messages: [] 01:50:10,148 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:50:10,149 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:50:10,149 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 prepared 01:50:10,149 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:10,149 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 prepared 01:50:10,149 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,149 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,149 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:10,149 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,149 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,149 PM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 113 01:50:10,149 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] 01:50:10,149 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] 01:50:10,149 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:10,149 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:10,149 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 01:50:10,149 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 submitted to persistence 01:50:10,149 PM [cluster-test-shard-dispatcher-23] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 282 01:50:10,149 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=282}] 01:50:10,149 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=282}] 01:50:10,149 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:10,149 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:10,149 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,149 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 submitted to persistence 01:50:10,149 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] 01:50:10,149 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=282}] 01:50:10,150 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}]]] 01:50:10,150 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:50:10,150 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:10,150 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 01:50:10,150 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 01:50:10,150 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}]]] 01:50:10,150 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:50:10,150 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:10,150 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 01:50:10,150 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 01:50:10,150 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 1 01:50:10,150 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:10,150 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 1 01:50:10,150 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:10,151 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 01:50:10,151 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 01:50:10,151 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-default-testSnapshotOnRootOverwrite status sync done true 01:50:10,152 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 0, nextIndex: 1 01:50:10,152 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 0, nextIndex: 1 01:50:10,152 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:10,152 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:10,152 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 0 01:50:10,152 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 0 01:50:10,152 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 01:50:10,152 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 01:50:10,152 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 01:50:10,152 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 01:50:10,152 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:50:10,152 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:50:10,152 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-1-shard-default-testSnapshotOnRootOverwrite: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} does not match any client, pre-disabling client. 01:50:10,152 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-1-shard-cars-testSnapshotOnRootOverwrite: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} does not match any client, pre-disabling client. 01:50:10,153 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, nanosAgo=6427825} is already disabled 01:50:10,153 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, nanosAgo=6693651} is already disabled 01:50:10,153 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:10,153 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:10,153 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 01:50:10,153 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,153 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}]]] 01:50:10,153 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=282}]]] 01:50:10,153 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 01:50:10,153 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 01:50:10,154 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}]]] 01:50:10,154 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:50:10,154 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:10,154 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=113} 01:50:10,154 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] 01:50:10,154 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:50:10,154 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 0 01:50:10,154 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:10,154 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:50:10,154 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 0 01:50:10,154 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 01:50:10,154 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 01:50:10,155 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:50:10,155 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-2-shard-default-testSnapshotOnRootOverwrite: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} does not match any client, pre-disabling client. 01:50:10,155 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=282}]]] 01:50:10,155 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:50:10,155 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:10,155 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=282} 01:50:10,155 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=282}] 01:50:10,155 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:50:10,155 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 0 01:50:10,155 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:10,155 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:50:10,155 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 0 01:50:10,155 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 01:50:10,155 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 01:50:10,155 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:50:10,155 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-2-shard-cars-testSnapshotOnRootOverwrite: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} does not match any client, pre-disabling client. 01:50:10,156 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] 01:50:10,156 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 01:50:10,156 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 1, nextIndex: 2 01:50:10,156 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=282}] 01:50:10,156 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:10,156 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 1 01:50:10,156 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 01:50:10,156 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 01:50:10,156 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 1 data Simple{size=113} 01:50:10,156 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,156 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:50:10,156 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:50:10,156 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-default-testSnapshotOnRootOverwrite%231960658105$a] 01:50:10,157 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-default-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-913523604] with name member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher 01:50:10,157 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:50:10,157 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 0 for term 4 01:50:10,157 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:10,157 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 01:50:10,157 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] and replicatedToAllIndex is 0 01:50:10,157 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=4, lastIndex=1, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:50:10,157 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 6 01:50:10,157 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:50:10,158 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 1, nextIndex: 2 01:50:10,158 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:10,158 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 1 01:50:10,158 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 01:50:10,158 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 1 data Simple{size=282} 01:50:10,158 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,158 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:50:10,158 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:50:10,158 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-225800510$b] 01:50:10,158 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-1921230787] with name member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher 01:50:10,159 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:50:10,159 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 0 for term 3 01:50:10,159 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 1 01:50:10,159 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:10,159 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:10,159 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 1 01:50:10,159 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 01:50:10,159 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 1 data Simple{size=113} 01:50:10,159 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 01:50:10,159 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=282}] and replicatedToAllIndex is 0 01:50:10,159 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=3, lastIndex=1, lastTerm=3, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:50:10,159 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 01:50:10,159 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,159 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:50:10,159 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-default-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#1163741002] with name member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher 01:50:10,159 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:50:10,159 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 0 for term 4 01:50:10,159 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#1163741002] 01:50:10,159 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 01:50:10,159 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,160 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] and replicatedToAllIndex is 0 01:50:10,160 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=4, lastIndex=1, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:50:10,160 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 01:50:10,160 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 21.96 μs 01:50:10,160 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:50:10,160 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#1163741002] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:10,160 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-913523604] 01:50:10,160 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: commit succeeded 01:50:10,160 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:50:10,160 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,160 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-1921230787] 01:50:10,160 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=4, lastAppliedIndex=1, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@732f1eb6, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:50:10,160 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 1 01:50:10,160 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 4 01:50:10,160 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,160 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 01:50:10,160 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 48.90 μs 01:50:10,160 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 122.6 μs 01:50:10,160 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,160 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,160 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,160 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-913523604] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:10,160 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-1921230787] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:10,160 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:50:10,160 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105)] for shard default, trying to use DirectTransactionContextWrapper 01:50:10,160 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 - Creating local component for shard default using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@131c21b0 01:50:10,160 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:50:10,160 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,160 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=3, lastAppliedIndex=1, lastAppliedTerm=3, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7a968b4, electionTerm=3, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:50:10,160 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 1 01:50:10,160 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 3 01:50:10,160 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=4, lastAppliedIndex=1, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@1634fe7a, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:50:10,160 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 1 01:50:10,160 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 4 01:50:10,160 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 6 01:50:10,160 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 6 01:50:10,161 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 01:50:10,161 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 01:50:10,161 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 01:50:10,161 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 01:50:10,161 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 01:50:10,161 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 01:50:10,161 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 01:50:10,161 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 01:50:10,161 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 01:50:10,161 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 01:50:10,161 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 1 01:50:10,161 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a journal response 01:50:10,161 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:10,161 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 1 01:50:10,161 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 01:50:10,161 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 1 data Simple{size=282} 01:50:10,162 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:50:10,162 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a journal response 01:50:10,162 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 01:50:10,162 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-32032823] with name member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher 01:50:10,162 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:50:10,162 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 0 for term 3 01:50:10,162 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 01:50:10,162 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=282}] and replicatedToAllIndex is 0 01:50:10,162 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=3, lastIndex=1, lastTerm=3, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:50:10,162 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 01:50:10,162 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:50:10,163 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-32032823] 01:50:10,163 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,163 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-32032823] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:10,163 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 93.28 μs 01:50:10,163 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1533125287] 01:50:10,163 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 01:50:10,163 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510] 01:50:10,163 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510] 01:50:10,164 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#367909887] 01:50:10,164 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 01:50:10,164 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:50:10,164 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#534475122] 01:50:10,164 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#534475122] 01:50:10,164 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=3, lastAppliedIndex=1, lastAppliedTerm=3, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6ea11ea9, electionTerm=3, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:50:10,164 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 1 01:50:10,164 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 3 01:50:10,164 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 01:50:10,164 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 01:50:10,164 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 01:50:10,164 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 01:50:10,170 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 01:50:10,371 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 01:50:10,372 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,372 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,372 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,372 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 Readying 1 components for commit 01:50:10,372 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 Readying transaction for shard cars 01:50:10,372 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 = [ 1 ] 01:50:10,372 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,373 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 canCommit 01:50:10,373 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 01:50:10,373 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 01:50:10,373 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 01:50:10,373 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 validated 01:50:10,373 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:10,373 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: success 01:50:10,373 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 01:50:10,373 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 01:50:10,373 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 candidate ready 01:50:10,373 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: canCommit - candidate rootPath: / 01:50:10,373 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: canCommit - messages: [] 01:50:10,373 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 01:50:10,373 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 prepared 01:50:10,373 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:10,373 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 01:50:10,373 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 01:50:10,374 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:50:10,374 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=222}] 01:50:10,374 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=222}] 01:50:10,374 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:10,374 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:10,374 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,374 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=222}]]] 01:50:10,374 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 submitted to persistence 01:50:10,376 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=222}]]] 01:50:10,376 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:50:10,376 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:50:10,376 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 01:50:10,376 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=222}] 01:50:10,376 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 1 01:50:10,376 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=2, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:10,376 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=222}] 01:50:10,377 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 2, nextIndex: 3 01:50:10,377 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:10,377 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=222}] 01:50:10,377 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:50:10,377 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 2 01:50:10,377 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 1 to 2 01:50:10,377 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 2 data Simple{size=222} 01:50:10,377 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 01:50:10,377 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: commit - successfulFromPrevious: [] 01:50:10,377 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 01:50:10,377 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-225800510$c] 01:50:10,377 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=2 01:50:10,378 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 successfully completed direct commit 01:50:10,378 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 committed successfully 01:50:10,378 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,378 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-3-0 01:50:10,378 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-3-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,378 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-3-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,378 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-3-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,378 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 91.67 μs 01:50:10,378 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 01:50:10,378 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,378 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,378 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,378 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 Readying 1 components for commit 01:50:10,378 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 Readying transaction for shard cars 01:50:10,378 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 = [ 1 ] 01:50:10,378 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,379 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 canCommit 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 validated 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: success 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 candidate ready 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: canCommit - candidate rootPath: / 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: canCommit - messages: [] 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: preCommit - successfulFromPrevious: [] 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 prepared 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=222}] 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=222}] 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=2, prevLogTerm=3, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=222}]]] 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 submitted to persistence 01:50:10,379 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=222}] 01:50:10,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=2, prevLogTerm=3, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=222}]]] 01:50:10,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:50:10,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:50:10,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 01:50:10,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=222}] 01:50:10,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:50:10,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 2 01:50:10,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:10,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 3 01:50:10,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 2 01:50:10,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 1 to 2 01:50:10,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 2 data Simple{size=222} 01:50:10,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 01:50:10,381 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,381 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 67.16 μs 01:50:10,382 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=222}] 01:50:10,382 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=2 01:50:10,382 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 3, nextIndex: 4 01:50:10,382 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 01:50:10,382 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 3 01:50:10,382 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 2 to 3 01:50:10,382 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 3 data Simple{size=222} 01:50:10,382 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 01:50:10,382 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: commit - successfulFromPrevious: [] 01:50:10,382 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 changing state from COMMIT_PENDING to COMMITTED 01:50:10,383 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-225800510$d] 01:50:10,383 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 2 for term 3 01:50:10,383 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 successfully completed direct commit 01:50:10,383 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:50:10,383 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 committed successfully 01:50:10,383 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,383 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=3 01:50:10,383 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-5-0 01:50:10,383 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-5-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,383 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-5-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,383 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-5-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,383 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 101.0 μs 01:50:10,383 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 01:50:10,383 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,383 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,383 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,383 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 Readying 1 components for commit 01:50:10,383 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 Readying transaction for shard cars 01:50:10,383 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 = [ 1 ] 01:50:10,383 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,384 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 canCommit 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 validated 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: success 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 candidate ready 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: canCommit - candidate rootPath: / 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: canCommit - messages: [] 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: preCommit - successfulFromPrevious: [] 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 prepared 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=222}] 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=222}] 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 2 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=3, prevLogTerm=3, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=222}]]] 01:50:10,384 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 submitted to persistence 01:50:10,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 3 01:50:10,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 01:50:10,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 3 01:50:10,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 2 to 3 01:50:10,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 3 data Simple{size=222} 01:50:10,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 01:50:10,386 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 2 for term 3 01:50:10,386 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,386 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=3, prevLogTerm=3, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=222}]]] 01:50:10,386 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:50:10,386 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 01:50:10,386 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 01:50:10,386 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=222}] 01:50:10,386 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:50:10,386 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 106.0 μs 01:50:10,386 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=4, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:10,386 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,386 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=222}] 01:50:10,386 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=222}] 01:50:10,386 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=3 01:50:10,387 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 4, nextIndex: 5 01:50:10,387 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 01:50:10,387 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 4 01:50:10,387 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 3 to 4 01:50:10,387 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 4 data Simple{size=222} 01:50:10,387 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 01:50:10,387 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: commit - successfulFromPrevious: [] 01:50:10,387 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 changing state from COMMIT_PENDING to COMMITTED 01:50:10,387 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-225800510$e] 01:50:10,388 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 3 for term 3 01:50:10,388 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:50:10,388 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=4 01:50:10,388 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 successfully completed direct commit 01:50:10,388 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 committed successfully 01:50:10,388 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,388 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-7-0 01:50:10,388 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-7-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,388 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-7-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,388 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-7-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,388 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 79.04 μs 01:50:10,388 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 01:50:10,388 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,388 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,388 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,388 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 Readying 1 components for commit 01:50:10,388 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 Readying transaction for shard cars 01:50:10,388 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 = [ 1 ] 01:50:10,389 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,389 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 canCommit 01:50:10,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 4 01:50:10,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4 01:50:10,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 4 01:50:10,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 3 to 4 01:50:10,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 4 data Simple{size=222} 01:50:10,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 01:50:10,390 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 3 for term 3 01:50:10,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=4 01:50:10,391 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 128.0 μs 01:50:10,391 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 01:50:10,391 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 01:50:10,391 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 01:50:10,391 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 validated 01:50:10,391 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:10,391 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: success 01:50:10,391 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 01:50:10,391 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 01:50:10,391 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 candidate ready 01:50:10,391 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: canCommit - candidate rootPath: / 01:50:10,391 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: canCommit - messages: [] 01:50:10,391 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: preCommit - successfulFromPrevious: [] 01:50:10,391 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 prepared 01:50:10,391 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:10,391 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 01:50:10,391 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 01:50:10,392 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:50:10,392 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=5, term=3, payload=Simple{size=222}] 01:50:10,392 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=3, payload=Simple{size=222}] 01:50:10,392 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:10,392 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 3 01:50:10,392 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,392 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=4, prevLogTerm=3, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=3, payload=Simple{size=222}]]] 01:50:10,392 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 submitted to persistence 01:50:10,392 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=4, prevLogTerm=3, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=3, payload=Simple{size=222}]]] 01:50:10,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:50:10,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 01:50:10,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 01:50:10,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=3, payload=Simple{size=222}] 01:50:10,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:50:10,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=5, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:10,393 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=5, term=3, payload=Simple{size=222}] 01:50:10,394 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=5, term=3, payload=Simple{size=222}] 01:50:10,394 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 5, nextIndex: 6 01:50:10,394 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 01:50:10,394 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 5 01:50:10,394 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 4 to 5 01:50:10,394 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 5 data Simple{size=222} 01:50:10,394 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 01:50:10,394 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: commit - successfulFromPrevious: [] 01:50:10,394 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 changing state from COMMIT_PENDING to COMMITTED 01:50:10,394 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-225800510$f] 01:50:10,395 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 4 for term 3 01:50:10,395 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 01:50:10,395 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=5 01:50:10,395 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 successfully completed direct commit 01:50:10,395 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 committed successfully 01:50:10,395 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,395 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-9-0 01:50:10,395 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-9-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,395 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-9-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,395 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-9-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,395 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 108.1 μs 01:50:10,396 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 01:50:10,396 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,396 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,396 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,396 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 Readying 1 components for commit 01:50:10,396 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 Readying transaction for shard cars 01:50:10,396 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 = [ 1 ] 01:50:10,396 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,396 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 canCommit 01:50:10,396 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 01:50:10,396 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 01:50:10,396 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 01:50:10,396 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 validated 01:50:10,396 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:10,396 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: success 01:50:10,396 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 01:50:10,396 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 01:50:10,396 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 candidate ready 01:50:10,396 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: canCommit - candidate rootPath: / 01:50:10,396 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: canCommit - messages: [] 01:50:10,396 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: preCommit - successfulFromPrevious: [] 01:50:10,396 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 prepared 01:50:10,396 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:10,396 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 01:50:10,396 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 01:50:10,397 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:50:10,397 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=6, term=3, payload=Simple{size=222}] 01:50:10,397 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=3, payload=Simple{size=222}] 01:50:10,397 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:10,397 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 6, leaderLastIndex: 6, leaderSnapShotIndex: 4 01:50:10,397 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 6 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,397 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=5, prevLogTerm=3, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=3, payload=Simple{size=222}]]] 01:50:10,397 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 submitted to persistence 01:50:10,397 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=6, term=3, payload=Simple{size=222}] 01:50:10,398 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 5 01:50:10,398 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5 01:50:10,398 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 5 01:50:10,398 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 4 to 5 01:50:10,398 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 5 data Simple{size=222} 01:50:10,398 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 01:50:10,398 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 4 for term 3 01:50:10,398 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=5, prevLogTerm=3, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=3, payload=Simple{size=222}]]] 01:50:10,398 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:50:10,398 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 5, entries to be added from: 0 01:50:10,398 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 01:50:10,398 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=3, payload=Simple{size=222}] 01:50:10,399 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:50:10,399 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=6, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:10,399 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,399 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 89.26 μs 01:50:10,399 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=6, term=3, payload=Simple{size=222}] 01:50:10,399 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=5 01:50:10,400 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 6 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,400 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 6, nextIndex: 7 01:50:10,400 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 5 01:50:10,400 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 6 01:50:10,400 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 5 to 6 01:50:10,400 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 6 data Simple{size=222} 01:50:10,400 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 01:50:10,400 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: commit - successfulFromPrevious: [] 01:50:10,400 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 changing state from COMMIT_PENDING to COMMITTED 01:50:10,400 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-225800510$g] 01:50:10,400 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 successfully completed direct commit 01:50:10,401 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 committed successfully 01:50:10,401 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,401 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 5 for term 3 01:50:10,401 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-11-0 01:50:10,401 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 01:50:10,401 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-11-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,401 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-11-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,401 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 59.56 μs 01:50:10,401 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=6 01:50:10,401 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-11-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,401 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 01:50:10,401 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,401 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,401 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,401 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 Readying 1 components for commit 01:50:10,401 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 Readying transaction for shard cars 01:50:10,401 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 = [ 1 ] 01:50:10,401 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,401 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 canCommit 01:50:10,402 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 01:50:10,402 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 01:50:10,402 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 01:50:10,402 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 validated 01:50:10,402 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:10,402 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: success 01:50:10,402 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 01:50:10,402 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 01:50:10,402 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 candidate ready 01:50:10,402 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: canCommit - candidate rootPath: / 01:50:10,402 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 6 01:50:10,402 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: canCommit - messages: [] 01:50:10,402 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 5, lastIndex(): 6 01:50:10,402 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: preCommit - successfulFromPrevious: [] 01:50:10,402 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 6 01:50:10,402 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 prepared 01:50:10,402 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 5 to 6 01:50:10,402 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:10,402 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 01:50:10,402 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 6 data Simple{size=222} 01:50:10,402 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 01:50:10,402 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 01:50:10,402 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:50:10,402 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=7, term=3, payload=Simple{size=222}] 01:50:10,403 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=3, payload=Simple{size=222}] 01:50:10,403 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:10,403 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: 5 01:50:10,403 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 7 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,403 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=6, prevLogTerm=3, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=3, payload=Simple{size=222}]]] 01:50:10,403 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 5 for term 3 01:50:10,403 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 submitted to persistence 01:50:10,403 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=6 01:50:10,403 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,403 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 71.92 μs 01:50:10,403 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=7, term=3, payload=Simple{size=222}] 01:50:10,404 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=6, prevLogTerm=3, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=3, payload=Simple{size=222}]]] 01:50:10,404 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:50:10,404 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 01:50:10,404 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 7 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,404 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 01:50:10,404 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=3, payload=Simple{size=222}] 01:50:10,404 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:50:10,404 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=7, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:10,405 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=7, term=3, payload=Simple{size=222}] 01:50:10,405 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 7, nextIndex: 8 01:50:10,405 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 6 01:50:10,405 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 7 01:50:10,405 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 6 to 7 01:50:10,405 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 7 data Simple{size=222} 01:50:10,405 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 01:50:10,405 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: commit - successfulFromPrevious: [] 01:50:10,405 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 changing state from COMMIT_PENDING to COMMITTED 01:50:10,405 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-225800510$h] 01:50:10,405 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 6 for term 3 01:50:10,405 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 01:50:10,405 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=7 01:50:10,406 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 successfully completed direct commit 01:50:10,406 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 committed successfully 01:50:10,406 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,406 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-13-0 01:50:10,406 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-13-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,406 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-13-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,406 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-13-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,406 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 62.01 μs 01:50:10,406 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 01:50:10,406 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,406 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,406 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,406 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 Readying 1 components for commit 01:50:10,406 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 Readying transaction for shard cars 01:50:10,406 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 = [ 1 ] 01:50:10,406 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,406 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 canCommit 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 01:50:10,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 7 01:50:10,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 7, context.getLastApplied(): 6, lastIndex(): 7 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 01:50:10,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 7 01:50:10,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 6 to 7 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 01:50:10,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 7 data Simple{size=222} 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 validated 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: success 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 01:50:10,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 candidate ready 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: canCommit - candidate rootPath: / 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: canCommit - messages: [] 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: preCommit - successfulFromPrevious: [] 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 prepared 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 01:50:10,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 6 for term 3 01:50:10,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=7 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=8, term=3, payload=Simple{size=222}] 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=3, payload=Simple{size=222}] 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0, logIndex: 8, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 8, leaderLastIndex: 8, leaderSnapShotIndex: 6 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 8 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=7, prevLogTerm=3, leaderCommit=7, replicatedToAllIndex=6, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=8, term=3, payload=Simple{size=222}]]] 01:50:10,407 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 submitted to persistence 01:50:10,408 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,408 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 51.67 μs 01:50:10,408 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=8, term=3, payload=Simple{size=222}] 01:50:10,408 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=7, prevLogTerm=3, leaderCommit=7, replicatedToAllIndex=6, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=8, term=3, payload=Simple{size=222}]]] 01:50:10,408 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:50:10,408 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 7, entries to be added from: 0 01:50:10,408 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 01:50:10,408 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 8 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,408 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=3, payload=Simple{size=222}] 01:50:10,408 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:50:10,408 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=8, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:10,409 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=8, term=3, payload=Simple{size=222}] 01:50:10,410 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 8, nextIndex: 9 01:50:10,410 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 8, lastAppliedIndex: 7 01:50:10,410 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 8 01:50:10,410 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 7 to 8 01:50:10,410 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 8 data Simple{size=222} 01:50:10,410 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 01:50:10,410 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: commit - successfulFromPrevious: [] 01:50:10,410 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 changing state from COMMIT_PENDING to COMMITTED 01:50:10,410 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-225800510$i] 01:50:10,411 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 7 for term 3 01:50:10,411 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: 7 01:50:10,411 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=8 01:50:10,411 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 successfully completed direct commit 01:50:10,411 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 committed successfully 01:50:10,411 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,411 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-15-0 01:50:10,411 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-15-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,411 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-15-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,411 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-15-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,411 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 104.1 μs 01:50:10,411 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 01:50:10,411 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,411 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,411 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,412 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 Readying 1 components for commit 01:50:10,412 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 Readying transaction for shard cars 01:50:10,412 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 = [ 1 ] 01:50:10,412 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,412 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 canCommit 01:50:10,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 8 01:50:10,412 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 01:50:10,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 8, context.getLastApplied(): 7, lastIndex(): 8 01:50:10,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 8 01:50:10,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 7 to 8 01:50:10,412 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 01:50:10,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 8 data Simple{size=222} 01:50:10,412 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 01:50:10,412 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 validated 01:50:10,412 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:10,412 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: success 01:50:10,412 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 01:50:10,412 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 01:50:10,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 01:50:10,412 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 candidate ready 01:50:10,412 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: canCommit - candidate rootPath: / 01:50:10,412 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: canCommit - messages: [] 01:50:10,412 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: preCommit - successfulFromPrevious: [] 01:50:10,412 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 prepared 01:50:10,412 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:10,412 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 01:50:10,412 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 01:50:10,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 7 for term 3 01:50:10,413 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=8 01:50:10,413 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:50:10,413 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=9, term=3, payload=Simple{size=222}] 01:50:10,413 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=3, payload=Simple{size=222}] 01:50:10,413 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0, logIndex: 9, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:10,413 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 9, leaderLastIndex: 9, leaderSnapShotIndex: 7 01:50:10,413 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 9 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,413 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=8, prevLogTerm=3, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=3, payload=Simple{size=222}]]] 01:50:10,413 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 submitted to persistence 01:50:10,413 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,413 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 63.92 μs 01:50:10,413 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=9, term=3, payload=Simple{size=222}] 01:50:10,414 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 9 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,414 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=8, prevLogTerm=3, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=3, payload=Simple{size=222}]]] 01:50:10,414 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:50:10,414 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 8, entries to be added from: 0 01:50:10,414 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 01:50:10,414 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=3, payload=Simple{size=222}] 01:50:10,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:50:10,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=9, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:10,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=9, term=3, payload=Simple{size=222}] 01:50:10,415 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 9, nextIndex: 10 01:50:10,415 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 9, lastAppliedIndex: 8 01:50:10,415 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 9 01:50:10,415 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 8 to 9 01:50:10,415 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 9 data Simple{size=222} 01:50:10,415 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 01:50:10,415 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: commit - successfulFromPrevious: [] 01:50:10,415 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 changing state from COMMIT_PENDING to COMMITTED 01:50:10,415 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-225800510$j] 01:50:10,416 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 8 for term 3 01:50:10,416 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 10, leaderLastIndex: 9, leaderSnapShotIndex: 8 01:50:10,416 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=9 01:50:10,416 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 committed successfully 01:50:10,416 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,416 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 successfully completed direct commit 01:50:10,416 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-17-0 01:50:10,416 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 83.35 μs 01:50:10,416 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-17-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,416 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-17-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,416 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-17-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,416 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 01:50:10,416 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,416 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,417 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,417 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 Readying 1 components for commit 01:50:10,417 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 Readying transaction for shard cars 01:50:10,417 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 = [ 1 ] 01:50:10,417 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,417 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 canCommit 01:50:10,417 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 01:50:10,417 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 01:50:10,417 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 01:50:10,417 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 validated 01:50:10,417 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:10,417 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: success 01:50:10,417 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 01:50:10,417 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 01:50:10,417 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 candidate ready 01:50:10,417 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: canCommit - candidate rootPath: / 01:50:10,417 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: canCommit - messages: [] 01:50:10,417 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: preCommit - successfulFromPrevious: [] 01:50:10,417 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 prepared 01:50:10,417 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:10,417 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 01:50:10,417 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 01:50:10,417 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 9 01:50:10,417 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 9, context.getLastApplied(): 8, lastIndex(): 9 01:50:10,417 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 9 01:50:10,417 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 8 to 9 01:50:10,417 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 9 data Simple{size=222} 01:50:10,417 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:50:10,417 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=10, term=3, payload=Simple{size=222}] 01:50:10,417 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=3, payload=Simple{size=222}] 01:50:10,417 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 01:50:10,418 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0, logIndex: 10, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:10,418 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 10, leaderLastIndex: 10, leaderSnapShotIndex: 8 01:50:10,418 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 10 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,418 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=9, prevLogTerm=3, leaderCommit=9, replicatedToAllIndex=8, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=10, term=3, payload=Simple{size=222}]]] 01:50:10,418 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 submitted to persistence 01:50:10,418 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,418 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 8 for term 3 01:50:10,418 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=9 01:50:10,418 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 59.92 μs 01:50:10,418 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=10, term=3, payload=Simple{size=222}] 01:50:10,420 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 10 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,420 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=9, prevLogTerm=3, leaderCommit=9, replicatedToAllIndex=8, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=10, term=3, payload=Simple{size=222}]]] 01:50:10,420 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:50:10,420 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 9, entries to be added from: 0 01:50:10,420 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 01:50:10,420 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=3, payload=Simple{size=222}] 01:50:10,420 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:50:10,420 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=10, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:10,421 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=10, term=3, payload=Simple{size=222}] 01:50:10,421 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 10, nextIndex: 11 01:50:10,422 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 10, lastAppliedIndex: 9 01:50:10,422 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 10 01:50:10,422 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 9 to 10 01:50:10,422 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 10 data Simple{size=222} 01:50:10,422 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 01:50:10,422 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: commit - successfulFromPrevious: [] 01:50:10,422 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 changing state from COMMIT_PENDING to COMMITTED 01:50:10,422 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-225800510$k] 01:50:10,422 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 9 for term 3 01:50:10,422 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 11, leaderLastIndex: 10, leaderSnapShotIndex: 9 01:50:10,422 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=10 01:50:10,422 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 successfully completed direct commit 01:50:10,422 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 committed successfully 01:50:10,422 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,422 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-19-0 01:50:10,423 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-19-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,423 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-19-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,423 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-19-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 124.2 μs 01:50:10,423 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 01:50:10,423 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,423 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,423 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,423 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 Readying 1 components for commit 01:50:10,423 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 Readying transaction for shard cars 01:50:10,423 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 = [ 1 ] 01:50:10,423 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,423 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 canCommit 01:50:10,424 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 10 01:50:10,424 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 01:50:10,424 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 10, context.getLastApplied(): 9, lastIndex(): 10 01:50:10,424 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 10 01:50:10,424 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 9 to 10 01:50:10,424 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 01:50:10,424 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 10 data Simple{size=222} 01:50:10,424 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 01:50:10,424 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 validated 01:50:10,424 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:10,424 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: success 01:50:10,424 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 01:50:10,424 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 01:50:10,424 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 01:50:10,424 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 candidate ready 01:50:10,424 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: canCommit - candidate rootPath: / 01:50:10,425 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: canCommit - messages: [] 01:50:10,425 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: preCommit - successfulFromPrevious: [] 01:50:10,425 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 prepared 01:50:10,425 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:10,425 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 01:50:10,425 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 01:50:10,425 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 9 for term 3 01:50:10,425 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=10 01:50:10,425 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:50:10,425 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=11, term=3, payload=Simple{size=222}] 01:50:10,425 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=3, payload=Simple{size=222}] 01:50:10,425 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0, logIndex: 11, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:10,425 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 11, leaderLastIndex: 11, leaderSnapShotIndex: 9 01:50:10,425 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 11 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,425 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=10, prevLogTerm=3, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=3, payload=Simple{size=222}]]] 01:50:10,425 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 submitted to persistence 01:50:10,426 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,426 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 75.94 μs 01:50:10,426 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=11, term=3, payload=Simple{size=222}] 01:50:10,427 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 11 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,428 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=10, prevLogTerm=3, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=3, payload=Simple{size=222}]]] 01:50:10,428 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:50:10,428 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 10, entries to be added from: 0 01:50:10,428 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 01:50:10,428 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=3, payload=Simple{size=222}] 01:50:10,428 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:50:10,428 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=11, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:10,428 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=11, term=3, payload=Simple{size=222}] 01:50:10,428 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 11, nextIndex: 12 01:50:10,429 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 11, lastAppliedIndex: 10 01:50:10,429 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 11 01:50:10,429 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 10 to 11 01:50:10,429 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 11 data Simple{size=222} 01:50:10,429 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 01:50:10,429 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: commit - successfulFromPrevious: [] 01:50:10,429 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 changing state from COMMIT_PENDING to COMMITTED 01:50:10,429 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-225800510$l] 01:50:10,429 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 10 for term 3 01:50:10,429 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 12, leaderLastIndex: 11, leaderSnapShotIndex: 10 01:50:10,429 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=11 01:50:10,429 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 successfully completed direct commit 01:50:10,429 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 committed successfully 01:50:10,429 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,430 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-21-0 01:50:10,430 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 96.43 μs 01:50:10,430 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-21-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,430 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-21-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,430 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-21-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,430 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1533125287] 01:50:10,430 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 01:50:10,430 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510] 01:50:10,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 11 01:50:10,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 11, context.getLastApplied(): 10, lastIndex(): 11 01:50:10,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 11 01:50:10,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 10 to 11 01:50:10,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 11 data Simple{size=222} 01:50:10,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 01:50:10,431 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 10 for term 3 01:50:10,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=11 01:50:10,432 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 67.02 μs 01:50:10,432 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510] 01:50:10,432 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#367909887] 01:50:10,432 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 01:50:10,432 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#534475122] 01:50:10,432 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#534475122] 01:50:10,486 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 closeTransaction called 01:50:10,486 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_9#-812413072)] 01:50:10,636 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,636 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,636 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,636 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,636 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105)] for shard default, trying to use DirectTransactionContextWrapper 01:50:10,636 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 - Creating local component for shard default using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@131c21b0 01:50:10,636 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,636 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 Readying 2 components for commit 01:50:10,636 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 Readying transaction for shard cars 01:50:10,636 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 = [ 1 ] 01:50:10,636 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,637 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 Readying transaction for shard default 01:50:10,637 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,637 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 = [ 1 ] 01:50:10,637 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105)] 01:50:10,637 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,637 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,637 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 canCommit 01:50:10,637 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,637 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransactionReadyReplyMapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 readyTransaction 01:50:10,637 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransactionReadyReplyMapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 readyTransaction 01:50:10,637 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] resolved 01:50:10,637 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 resolved to actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,637 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105)] resolved 01:50:10,637 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: successfully resolved all cohort actors 01:50:10,637 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 finishCanCommit 01:50:10,637 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: sending CanCommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,637 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,637 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 resolved to actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105)] 01:50:10,637 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Can committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,637 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,638 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 validated 01:50:10,638 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:10,638 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: success 01:50:10,638 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:50:10,638 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: sending CanCommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105)] 01:50:10,638 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105)] 01:50:10,638 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Can committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,638 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,638 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 validated 01:50:10,638 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:10,638 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: success 01:50:10,638 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:50:10,638 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit returning result: true 01:50:10,638 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 commit 01:50:10,638 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 finish commit 01:50:10,638 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Sending CommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,638 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] 01:50:10,638 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Sending CommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105)] 01:50:10,638 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105)] 01:50:10,638 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 candidate ready 01:50:10,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit - candidate rootPath: / 01:50:10,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 candidate ready 01:50:10,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit - candidate rootPath: / 01:50:10,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit - messages: [] 01:50:10,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: preCommit - successfulFromPrevious: [] 01:50:10,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 prepared 01:50:10,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:10,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit - messages: [] 01:50:10,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: preCommit - successfulFromPrevious: [] 01:50:10,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 prepared 01:50:10,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:10,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 114 01:50:10,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] 01:50:10,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] 01:50:10,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 283 01:50:10,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:10,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=12, term=3, payload=Simple{size=283}] 01:50:10,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=12, term=3, payload=Simple{size=283}] 01:50:10,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, logIndex: 12, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:10,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:10,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 2 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 01:50:10,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}]]] 01:50:10,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 12, leaderLastIndex: 12, leaderSnapShotIndex: 10 01:50:10,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 12 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:10,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=11, prevLogTerm=3, leaderCommit=11, replicatedToAllIndex=10, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=12, term=3, payload=Simple{size=283}]]] 01:50:10,639 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 submitted to persistence 01:50:10,639 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 submitted to persistence 01:50:10,640 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] 01:50:10,640 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=12, term=3, payload=Simple{size=283}] 01:50:10,641 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}]]] 01:50:10,641 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:50:10,641 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:50:10,641 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=114} 01:50:10,641 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] 01:50:10,642 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 1 01:50:10,642 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:10,642 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=11, prevLogTerm=3, leaderCommit=11, replicatedToAllIndex=10, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=12, term=3, payload=Simple{size=283}]]] 01:50:10,642 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:50:10,642 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 11, entries to be added from: 0 01:50:10,642 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=283} 01:50:10,642 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=12, term=3, payload=Simple{size=283}] 01:50:10,642 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:50:10,642 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=12, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:10,642 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] 01:50:10,642 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=12, term=3, payload=Simple{size=283}] 01:50:10,643 PM [cluster-test-shard-dispatcher-29] [WARN] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=12, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 204, lastApplied : 11, commitIndex : 11 01:50:10,643 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 12, nextIndex: 13 01:50:10,643 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 12, lastAppliedIndex: 11 01:50:10,643 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 2, nextIndex: 3 01:50:10,643 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 12 01:50:10,643 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 11 to 12 01:50:10,643 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:50:10,643 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 12 data Simple{size=283} 01:50:10,643 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 2 01:50:10,643 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 1 to 2 01:50:10,643 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,643 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 2 data Simple{size=114} 01:50:10,643 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,643 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: commit - successfulFromPrevious: [] 01:50:10,643 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: commit - successfulFromPrevious: [] 01:50:10,643 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from COMMIT_PENDING to COMMITTED 01:50:10,643 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from COMMIT_PENDING to COMMITTED 01:50:10,643 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-225800510$m] 01:50:10,643 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-default-testSnapshotOnRootOverwrite%231960658105$n] 01:50:10,643 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:50:10,643 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:10,643 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=2 01:50:10,644 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] and replicatedToAllIndex is 1 01:50:10,644 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:50:10,644 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 11 for term 3 01:50:10,644 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=4, lastIndex=2, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:50:10,644 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 9 01:50:10,644 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 13, leaderLastIndex: 12, leaderSnapShotIndex: 11 01:50:10,644 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=12 01:50:10,644 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:50:10,644 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=12, term=3, payload=Simple{size=283}] and replicatedToAllIndex is 11 01:50:10,644 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=12, lastAppliedTerm=3, lastIndex=12, lastTerm=3, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:50:10,644 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 28 01:50:10,644 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:50:10,644 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,644 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: commit succeeded 01:50:10,644 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 105.6 μs 01:50:10,644 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,644 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:50:10,645 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 93.26 μs 01:50:10,645 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=2, lastTerm=4, lastAppliedIndex=2, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@78cd903d, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:50:10,645 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 2 01:50:10,645 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 2 and term: 4 01:50:10,645 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:50:10,645 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=12, lastTerm=3, lastAppliedIndex=12, lastAppliedTerm=3, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@3eff191a, electionTerm=3, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:50:10,645 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 12 01:50:10,645 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 12 and term: 3 01:50:10,645 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0 01:50:10,645 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:10,645 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@432f835 01:50:10,645 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,645 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1960658105)] for shard default, trying to use DirectTransactionContextWrapper 01:50:10,645 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0 - Creating local component for shard default using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@131c21b0 01:50:10,645 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:10,646 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1533125287] 01:50:10,646 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 01:50:10,646 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510] 01:50:10,647 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510] 01:50:10,647 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#367909887] 01:50:10,647 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 01:50:10,647 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#534475122] 01:50:10,647 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#534475122] 01:50:10,647 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 9 01:50:10,647 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 9 01:50:10,647 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 28 01:50:10,647 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 28 01:50:10,647 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 01:50:10,647 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 01:50:10,648 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 01:50:10,648 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 01:50:10,648 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a journal response 01:50:10,648 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 01:50:10,648 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 2 01:50:10,648 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:50:10,648 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 2 01:50:10,648 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 1 to 2 01:50:10,649 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 2 data Simple{size=114} 01:50:10,649 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,649 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,649 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:50:10,649 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=2 01:50:10,649 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 28.63 μs 01:50:10,649 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] and replicatedToAllIndex is 1 01:50:10,649 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=4, lastIndex=2, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:50:10,649 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 7 01:50:10,649 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:50:10,649 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:50:10,649 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=2, lastTerm=4, lastAppliedIndex=2, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@713251f3, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:50:10,649 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 2 01:50:10,649 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 12 01:50:10,649 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 2 and term: 4 01:50:10,649 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 12, context.getLastApplied(): 11, lastIndex(): 12 01:50:10,649 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 12 01:50:10,649 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 11 to 12 01:50:10,649 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 7 01:50:10,649 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 12 data Simple{size=283} 01:50:10,649 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 7 01:50:10,649 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 01:50:10,650 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 01:50:10,650 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 01:50:10,650 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:50:10,650 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:50:10,650 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 11 for term 3 01:50:10,650 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 92.20 μs 01:50:10,650 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=12 01:50:10,650 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=12, term=3, payload=Simple{size=283}] and replicatedToAllIndex is 11 01:50:10,651 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=12, lastAppliedTerm=3, lastIndex=12, lastTerm=3, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:50:10,651 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 27 01:50:10,651 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:50:10,651 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:50:10,651 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=12, lastTerm=3, lastAppliedIndex=12, lastAppliedTerm=3, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2169aa3f, electionTerm=3, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:50:10,651 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 12 01:50:10,651 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 12 and term: 3 01:50:10,651 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a journal response 01:50:10,651 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 27 01:50:10,651 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 27 01:50:10,651 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 01:50:10,651 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 01:50:10,662 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 01:50:10,698 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#534475122] 01:50:10,901 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:50:10,902 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-1-shard-cars-testSnapshotOnRootOverwrite 01:50:10,902 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-1-shard-default-testSnapshotOnRootOverwrite 01:50:10,902 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSnapshotOnRootOverwrite - waiting on 2 shards 01:50:10,902 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onShutDown 01:50:10,902 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating leader transfer 01:50:10,902 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onShutDown 01:50:10,902 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating leader transfer 01:50:10,902 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6633ce74 01:50:10,902 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2fb0b1e2 01:50:10,902 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6633ce74 01:50:10,902 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2fb0b1e2 01:50:10,902 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 01:50:10,902 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Attempting to transfer leadership 01:50:10,902 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Attempting to transfer leadership 01:50:10,902 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 2 01:50:10,902 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 13, leaderLastIndex: 12, leaderSnapShotIndex: 12 01:50:10,902 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:10,902 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:10,902 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 01:50:10,902 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 01:50:10,904 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 01:50:10,904 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite] 01:50:10,905 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite] 01:50:10,905 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 01:50:10,905 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:10,905 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:10,905 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 01:50:10,905 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 01:50:10,905 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 01:50:10,905 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSnapshotOnRootOverwrite, matchIndex: 12, lastIndex: 12, isVoting: true 01:50:10,905 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-default-testSnapshotOnRootOverwrite, matchIndex: 2, lastIndex: 2, isVoting: true 01:50:10,905 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Follower's log matches - sending ElectionTimeout 01:50:10,905 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Follower's log matches - sending ElectionTimeout 01:50:10,905 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 13, leaderLastIndex: 12, leaderSnapShotIndex: 12 01:50:10,905 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 2 01:50:10,906 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Leader transfer complete 01:50:10,906 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Leader transfer complete 01:50:10,928 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:50:11,318 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All shards shutdown - sending PoisonPill to self 01:50:11,318 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] ShardManager - shard-manager-testSnapshotOnRootOverwrite: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-225800510]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:11,319 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSnapshotOnRootOverwrite 01:50:11,319 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader)_slicer-id-444: Closing 01:50:11,319 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 01:50:11,319 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-1-shard-default-testSnapshotOnRootOverwrite 01:50:11,319 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-default-testSnapshotOnRootOverwrite (Leader)_slicer-id-445: Closing 01:50:11,319 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 01:50:11,319 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-testSnapshotOnRootOverwrite 01:50:11,320 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 01:50:11,320 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:50:11,320 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: shutting down 01:50:11,320 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 01:50:11,320 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 01:50:11,320 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1533125287] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1533125287] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:11,321 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:11,321 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/11/2024 13:50:11.333] [cluster-test-akka.remote.default-remote-dispatcher-5] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-713)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:50:11.333] [cluster-test-akka.remote.default-remote-dispatcher-5] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-713)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:50:11.334] [cluster-test-akka.remote.default-remote-dispatcher-5] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-713)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:50:11.334] [cluster-test-akka.remote.default-remote-dispatcher-5] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-713)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:11,350 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSnapshotOnRootOverwrite: Closing 01:50:11,351 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSnapshotOnRootOverwrite 01:50:11,351 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Closing 01:50:11,351 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 01:50:11,351 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-2-shard-default-testSnapshotOnRootOverwrite 01:50:11,351 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Closing 01:50:11,351 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 01:50:11,351 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testSnapshotOnRootOverwrite 01:50:11,352 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:11,352 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:12,346 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4924890193741761073] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:12,366 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-4544008715493771672] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:12,374 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:12,374 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:12,375 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:12.377] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:12,383 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:12,383 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:12,385 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:12.386] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:12,403 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:50:12,412 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [6340192823465897165] 01:50:12,412 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:12,414 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:12,414 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:12,414 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:12,414 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:12,426 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:12,426 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:12,426 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:12,431 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:12,437 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [140965883537667647] 01:50:12,437 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:12,438 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:12,438 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:12,438 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:12,438 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:12,452 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:12,462 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [1506017038577935515] 01:50:12,462 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:12,465 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:12,465 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:12,468 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:12,468 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:12,474 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:12,476 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:12,477 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 01:50:12,488 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-2095005314] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,490 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: no initial generation override, starting from 0 01:50:12,490 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:50:12,490 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7b1322d3 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@496f57fc 01:50:12,490 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4d50ca09 01:50:12,495 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:12,497 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:12,500 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1710165012499, None)) 01:50:12,500 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:12,501 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165012498,0,0)) 01:50:12,501 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@496f57fc to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@d1e0b52 01:50:12,501 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} started 01:50:12,502 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:12,503 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 01:50:12,504 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:12,505 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 01:50:12,508 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:12,508 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#1304697086] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,509 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 01:50:12,510 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 01:50:12,510 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-1-shard-cars-testWriteTransactionWithSingleShard 01:50:12,511 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:12,511 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:50:12,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@782b7687 01:50:12,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:12,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testWriteTransactionWithSingleShard 01:50:12,511 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:50:12,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:50:12,511 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard on shard member-1-shard-cars-testWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-548285848] 01:50:12,512 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-cars-testWriteTransactionWithSingleShard, persistent : true 01:50:12,512 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 01:50:12,513 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-548285848] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,513 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-notifier#1902563475 created and ready for shard:member-1-shard-cars-testWriteTransactionWithSingleShard 01:50:12,513 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-member-1:snapshot-read#-1197345874] 01:50:12,513 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-notifier#1902563475] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,513 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-member-1:snapshot-read#-1197345874] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,513 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:12,513 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:12,513 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 01:50:12,513 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-1-shard-cars-testWriteTransactionWithSingleShard is initialized 01:50:12,513 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:12,514 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 01:50:12,514 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 01:50:12,514 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard 01:50:12,514 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from null to Follower 01:50:12,521 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: no initial generation override, starting from 0 01:50:12,521 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:50:12,521 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@52c6d1e1 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@bba6f3e 01:50:12,521 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 01:50:12,521 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4b25dacf 01:50:12,521 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:12,521 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1710165012521, None)) 01:50:12,521 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:12,521 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:50:12,522 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165012520,0,0)) 01:50:12,522 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7a46fad0 01:50:12,522 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@bba6f3e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@78e3217 01:50:12,522 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} started 01:50:12,523 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:12,523 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:12,523 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:50:12,523 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:12,523 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard on shard member-2-shard-cars-testWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-939591210] 01:50:12,524 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithSingleShard, persistent : true 01:50:12,525 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 01:50:12,525 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-939591210] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,525 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#526778487] 01:50:12,525 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-1029768159 created and ready for shard:member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:12,525 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-1029768159] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,525 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#526778487] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:12,526 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:12,526 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:12,526 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 01:50:12,526 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithSingleShard is initialized 01:50:12,526 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:12,526 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 01:50:12,526 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:50:12,526 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 01:50:12,526 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard 01:50:12,526 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testWriteTransactionWithSingleShard from null to Follower 01:50:12,573 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-2095005314] 01:50:12,573 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 01:50:12,573 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-548285848] 01:50:12,751 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:12,751 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 01:50:12,751 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testWriteTransactionWithSingleShard] 01:50:12,751 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:50:12,751 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 1 01:50:12,751 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:12,751 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 01:50:12,751 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:12,751 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 01:50:12,751 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 01:50:12,752 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from Follower to Candidate 01:50:13,0 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:50:13,0 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:50:13,0 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 2 01:50:13,116 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:13,310 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:50:13,310 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:50:13,310 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 3 01:50:13,432 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:13,432 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:13,432 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:13,432 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard 01:50:13,432 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:50:13,432 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:13,432 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:13,432 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:50:13,432 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-2-shard-cars-testWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:13,446 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:13,446 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard 01:50:13,446 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:13,446 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:13,446 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:13,447 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:13,481 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:13,580 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:50:13,580 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:50:13,580 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 4 01:50:13,581 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:13,582 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:13,582 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Cluster state: [] 01:50:13,582 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:13,582 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:13,582 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 01:50:13,583 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:13,583 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:50:13,583 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:13,584 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:13,584 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:13,584 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testWriteTransactionWithSingleShard] 01:50:13,584 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:13,585 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:13,585 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:50:13,585 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: became leader with frontend state for [] 01:50:13,585 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 01:50:13,585 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:50:13,585 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:13,585 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:13,585 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from Candidate to Leader 01:50:13,585 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 01:50:13,586 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard: Closing 01:50:13,586 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testWriteTransactionWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:50:13,586 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithSingleShard status sync done false 01:50:13,586 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:50:13,586 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:13,586 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:13,587 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 01:50:13,587 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: -1, nextIndex: 0 01:50:13,587 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:13,630 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testWriteTransactionWithSingleShard commit rateLimit adjusted to 162.12298098119095 01:50:13,631 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:13,631 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#1304697086] 01:50:13,631 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:50:13,631 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:50:13,631 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:50:13,631 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:50:13,631 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Readying 1 components for commit 01:50:13,631 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Readying transaction for shard cars 01:50:13,631 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:50:13,631 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 canCommit 01:50:13,631 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:13,631 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard, primaryVersion=13] 01:50:13,631 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard, primaryVersion=13] 01:50:13,632 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars 01:50:13,632 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - creating WRITE_ONLY transaction context 01:50:13,632 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:13,632 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:50:13,632 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 01:50:13,632 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 01:50:13,632 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 directCommit called 01:50:13,632 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 sending 4 batched modifications, ready: true 01:50:13,632 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 01:50:13,634 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, disabling transaction tracking 01:50:13,635 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 01:50:13,635 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 01:50:13,635 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:13,635 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:13,635 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:13,635 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}]]] 01:50:13,635 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying 4 batched modifications for Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:13,636 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Readying Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 of 4 operations, client version 13 01:50:13,636 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: readying transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:13,636 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 ready 01:50:13,636 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:13,636 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 validated 01:50:13,636 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:13,636 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: canCommit for member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: success 01:50:13,636 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Committing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:13,636 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:13,636 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 candidate ready 01:50:13,636 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:50:13,636 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: canCommit - messages: [] 01:50:13,636 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:50:13,636 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 prepared 01:50:13,636 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:13,636 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Finishing commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:13,636 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:13,637 PM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 245 01:50:13,637 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 01:50:13,637 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 01:50:13,637 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:13,637 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:13,637 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:13,637 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 submitted to persistence 01:50:13,637 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 01:50:13,638 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 01:50:13,638 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}]]] 01:50:13,638 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 01:50:13,638 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:13,638 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62} 01:50:13,638 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 01:50:13,638 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 1 01:50:13,638 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:13,639 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 01:50:13,639 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithSingleShard status sync done true 01:50:13,639 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 0, nextIndex: 1 01:50:13,639 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:13,639 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 0 01:50:13,639 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 01:50:13,639 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62} 01:50:13,639 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:50:13,639 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-1-shard-cars-testWriteTransactionWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} does not match any client, pre-disabling client. 01:50:13,639 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, nanosAgo=4805923} is already disabled 01:50:13,639 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:13,639 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:13,639 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}]]] 01:50:13,639 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 01:50:13,640 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}]]] 01:50:13,640 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 01:50:13,640 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:13,640 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=245} 01:50:13,640 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 01:50:13,640 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 2 01:50:13,640 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 0 01:50:13,640 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:13,640 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:50:13,640 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 0 01:50:13,640 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 01:50:13,640 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62} 01:50:13,640 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:50:13,640 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} does not match any client, pre-disabling client. 01:50:13,641 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 01:50:13,641 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 01:50:13,641 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 1, nextIndex: 2 01:50:13,641 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:13,641 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 1 01:50:13,641 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 01:50:13,641 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 1 data Simple{size=245} 01:50:13,641 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:13,641 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:50:13,641 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:50:13,641 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testWriteTransactionWithSingleShard_member-1-shard-cars-testWriteTransactionWithSingleShard$a] 01:50:13,641 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#387936897] with name member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher 01:50:13,641 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#387936897] 01:50:13,642 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 01:50:13,642 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:13,642 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:50:13,642 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 01:50:13,642 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 130.8 μs 01:50:13,642 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#387936897] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:13,644 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 modifications completed with CommitTransactionReply [version=13] 01:50:13,644 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 successfully completed direct commit 01:50:13,644 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 01:50:13,644 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:13,644 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:13,644 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - trying create transaction 01:50:13,644 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 01:50:13,644 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 Queuing TransactionOperation 01:50:13,644 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 1 01:50:13,644 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:13,644 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 1 01:50:13,644 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 01:50:13,644 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 1 data Simple{size=245} 01:50:13,644 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:13,645 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#1640456520] with name member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher 01:50:13,645 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 01:50:13,645 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 01:50:13,645 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#1640456520] 01:50:13,645 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:50:13,645 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 62.17 μs 01:50:13,645 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#1640456520] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:13,646 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 01:50:13,647 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_10#-120739951, transactionId=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0, version=13] 01:50:13,647 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:50:13,647 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_10#-120739951)] 01:50:13,649 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_10#-120739951] 01:50:13,649 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_10#-120739951] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:13,651 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 ReadData operation succeeded 01:50:13,651 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:13,651 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-2095005314] 01:50:13,651 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:50:13,651 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:13,651 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-548285848, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@247e8ecc] 01:50:13,651 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-548285848, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@247e8ecc] 01:50:13,651 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-548285848)] for shard cars 01:50:13,651 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 01:50:13,651 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@523e8aea 01:50:13,651 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:50:13,651 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:13,651 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:13,651 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - creating WRITE_ONLY transaction context 01:50:13,652 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 executeDelete called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 01:50:13,652 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 Readying 1 components for commit 01:50:13,652 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 Readying transaction for shard cars 01:50:13,652 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 directCommit called 01:50:13,652 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 sending 1 batched modifications, ready: true 01:50:13,652 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 01:50:13,652 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 canCommit 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying 1 batched modifications for Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Readying Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 of 1 operations, client version 13 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: readying transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 ready 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 validated 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: canCommit for member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: success 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Committing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 candidate ready 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: canCommit - candidate rootPath: / 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: canCommit - messages: [] 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 prepared 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Finishing commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 212 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}]]] 01:50:13,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 submitted to persistence 01:50:13,655 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}]]] 01:50:13,655 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 01:50:13,655 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:50:13,655 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=212} 01:50:13,655 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 01:50:13,655 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 2 01:50:13,655 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:13,656 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 2, nextIndex: 3 01:50:13,656 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:50:13,656 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 01:50:13,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 01:50:13,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:50:13,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 2 01:50:13,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 01:50:13,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 2 data Simple{size=212} 01:50:13,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:50:13,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: commit - successfulFromPrevious: [] 01:50:13,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 01:50:13,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testWriteTransactionWithSingleShard_member-1-shard-cars-testWriteTransactionWithSingleShard$d] 01:50:13,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 1 for term 4 01:50:13,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=2 01:50:13,659 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:50:13,659 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 60.30 μs 01:50:13,660 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 modifications completed with CommitTransactionReply [version=13] 01:50:13,660 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 successfully completed direct commit 01:50:13,660 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 01:50:13,660 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:13,660 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:13,660 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - trying create transaction 01:50:13,660 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 01:50:13,660 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 Queuing TransactionOperation 01:50:13,662 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 01:50:13,662 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_11#1506371316] 01:50:13,663 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_11#1506371316] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:13,663 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_11#1506371316, transactionId=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0, version=13] 01:50:13,663 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 executeRead DataExists called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 01:50:13,663 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.DataExists to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_11#1506371316)] 01:50:13,665 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 DataExists operation succeeded 01:50:13,665 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 01:50:13,665 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:13,665 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:13,665 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - trying create transaction 01:50:13,665 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 01:50:13,665 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 Queuing TransactionOperation 01:50:13,666 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 01:50:13,666 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_12#2077972844] 01:50:13,666 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_12#2077972844] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:13,667 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_12#2077972844, transactionId=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0, version=13] 01:50:13,667 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:50:13,667 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_12#2077972844)] 01:50:13,669 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 ReadData operation succeeded 01:50:13,669 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 01:50:13,669 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-548285848)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:13,669 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@523e8aea 01:50:13,669 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:13,669 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-2095005314] 01:50:13,669 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 01:50:13,669 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-548285848] 01:50:13,670 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-548285848] 01:50:13,823 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 2 01:50:13,823 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:50:13,823 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 2 01:50:13,823 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 01:50:13,823 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 2 data Simple{size=212} 01:50:13,823 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:50:13,824 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:50:13,824 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 1 for term 4 01:50:13,824 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=2 01:50:13,824 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 186.7 μs 01:50:13,923 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:50:13,924 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-1-shard-cars-testWriteTransactionWithSingleShard 01:50:13,924 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader)_slicer-id-450: Closing 01:50:13,924 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:50:13,924 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 01:50:13,925 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:13,925 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:13,932 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:13,932 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:13,932 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:13,932 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:13,952 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:50:13,952 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:13,952 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 01:50:13,952 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:50:13,952 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 01:50:13,953 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:13,953 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:14,942 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [6340192823465897165] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:14,971 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [140965883537667647] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:14,971 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:14,971 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:14,974 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:14.976] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:14,997 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:15,6 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://reinstated-member2@127.0.0.1:2559] with UID [3501411828807978588] 01:50:15,6 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:15,8 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:15,8 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Started up successfully 01:50:15,9 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:15,9 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:15,18 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:15,20 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:15,22 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 01:50:15,31 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2033407108] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:15,32 PM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 01:50:15,34 PM [reinstated-member2-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: recovered identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:50:15,40 PM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 01:50:15,40 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:15,40 PM [reinstated-member2-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 01:50:15,40 PM [reinstated-member2-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5e22893b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@277307d7 01:50:15,40 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-people-testWriteTransactionWithSingleShard 01:50:15,41 PM [reinstated-member2-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@38ac56ad 01:50:15,41 PM [reinstated-member2-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1710165015040, None)) 01:50:15,41 PM [reinstated-member2-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165015039,0,0)) 01:50:15,42 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4abcaf5b 01:50:15,42 PM [reinstated-member2-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@277307d7 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@233d5c15 01:50:15,42 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} started 01:50:15,42 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:15,42 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:15,42 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testWriteTransactionWithSingleShard 01:50:15,43 PM [reinstated-member2-shard-dispatcher-12] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithSingleShard, persistent : true 01:50:15,43 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-598997683] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:15,43 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard#1339892678] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:15,43 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Shard created : member-2-shard-people-testWriteTransactionWithSingleShard, persistent : true 01:50:15,43 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-911839670 created and ready for shard:member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:15,44 PM [reinstated-member2-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-567897575] 01:50:15,44 PM [reinstated-member2-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-911839670] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:15,44 PM [reinstated-member2-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard-notifier#149524206 created and ready for shard:member-2-shard-people-testWriteTransactionWithSingleShard 01:50:15,44 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-567897575] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:15,44 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard-notifier#149524206] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:15,46 PM [reinstated-member2-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 01:50:15,46 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-2-shard-people-testWriteTransactionWithSingleShard with journal batch size 1 01:50:15,46 PM [reinstated-member2-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-1964616969] 01:50:15,47 PM [reinstated-member2-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-1964616969] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:15,53 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://reinstated-member2/deadLetters] 01:50:15,53 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://reinstated-member2/deadLetters] 01:50:15,53 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 01:50:15,53 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:15,53 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://reinstated-member2/deadLetters] 01:50:15,53 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:50:15,53 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 01:50:15,53 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 0, size: 62 01:50:15,53 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 01:50:15,53 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 1, size: 245 01:50:15,53 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 01:50:15,53 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 0 to 0 01:50:15,53 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:15,53 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 01:50:15,54 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithSingleShard is no longer the leader 01:50:15,54 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: ignoring unhandled payload DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62} 01:50:15,54 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 01:50:15,54 PM [reinstated-member2-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from null to Follower 01:50:15,54 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Follower): Received TimeoutNow - switching to Candidate 01:50:15,54 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [] 01:50:15,54 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-2-shard-people-testWriteTransactionWithSingleShard 01:50:15,54 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Starting new election term 1 01:50:15,54 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:15,54 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithSingleShard (Follower): Closing 01:50:15,54 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:15,54 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithSingleShard is no longer the leader 01:50:15,54 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 01:50:15,54 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [people] 01:50:15,54 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-people-testWriteTransactionWithSingleShard is initialized 01:50:15,54 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 01:50:15,54 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 1 to 1 01:50:15,54 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 01:50:15,55 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@3914a55b}, purged=MutableUnsignedLongSet{size=0}} 01:50:15,55 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:50:15,55 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 01:50:15,55 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 01:50:15,55 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 2, size: 212 01:50:15,55 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 01:50:15,55 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 2 to 2 01:50:15,55 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 01:50:15,55 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:50:15,55 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 01:50:15,55 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://reinstated-member2/deadLetters] 01:50:15,56 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Recovery completed in 1.979 ms - Switching actor to Follower - last log index = 2, last log term = 4, snapshot index = -1, snapshot term = -1, journal size = 3 01:50:15,56 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:15,56 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 01:50:15,56 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Received TimeoutNow - switching to Candidate 01:50:15,56 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [] 01:50:15,56 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=5, votedFor=member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:15,56 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 5 01:50:15,56 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:50:15,56 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 01:50:15,56 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:15,56 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 01:50:15,56 PM [reinstated-member2-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 01:50:15,56 PM [reinstated-member2-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 01:50:15,61 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:50:15,61 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [] 01:50:15,61 PM [reinstated-member2-shard-dispatcher-12] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 01:50:15,61 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:50:15,61 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: became leader with frontend state for [] 01:50:15,61 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 01:50:15,61 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , registered listener akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard 01:50:15,61 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:50:15,62 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [] 01:50:15,62 PM [reinstated-member2-shard-dispatcher-12] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:50:15,62 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:50:15,62 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: became leader with frontend state for [member-2-frontend-datastore-testWriteTransactionWithSingleShard] 01:50:15,62 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 01:50:15,62 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 01:50:15,62 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithSingleShard is initialized 01:50:15,62 PM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithSingleShard, leaderId=member-2-shard-people-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:50:15,62 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:15,62 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:15,62 PM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-people-testWriteTransactionWithSingleShard from Candidate to Leader 01:50:15,62 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard 01:50:15,62 PM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:50:15,62 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:15,62 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:15,62 PM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testWriteTransactionWithSingleShard from Candidate to Leader 01:50:15,62 PM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 01:50:15,92 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2033407108] 01:50:15,92 PM [reinstated-member2-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 01:50:15,92 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-598997683] 01:50:15,93 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 01:50:15,93 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2033407108] 01:50:15,93 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 Queuing TransactionOperation 01:50:15,93 PM [reinstated-member2-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:15,93 PM [reinstated-member2-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-598997683, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@7606a553] 01:50:15,93 PM [reinstated-member2-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-598997683, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@7606a553] 01:50:15,93 PM [reinstated-member2-akka.actor.default-dispatcher-20] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0: Found primary ActorSelection[Anchor(akka://reinstated-member2/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-598997683)] for shard cars 01:50:15,93 PM [reinstated-member2-akka.actor.default-dispatcher-20] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 01:50:15,93 PM [reinstated-member2-akka.actor.default-dispatcher-20] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@595ce0a7 01:50:15,94 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 01:50:15,94 PM [reinstated-member2-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:15,94 PM [reinstated-member2-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-people-testWriteTransactionWithSingleShard 01:50:15,94 PM [reinstated-member2-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithSingleShard - waiting on 2 shards 01:50:15,94 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onShutDown 01:50:15,94 PM [reinstated-member2-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onShutDown 01:50:15,94 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@2d61b84c 01:50:15,94 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@2d61b84c 01:50:15,94 PM [reinstated-member2-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@5765652a 01:50:15,94 PM [reinstated-member2-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-people-testWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@5765652a 01:50:15,95 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - Stopping Shard member-2-shard-people-testWriteTransactionWithSingleShard 01:50:15,95 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] MessageSlicer - member-2-shard-people-testWriteTransactionWithSingleShard (Leader)_slicer-id-454: Closing 01:50:15,95 PM [reinstated-member2-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithSingleShard 01:50:15,95 PM [reinstated-member2-shard-dispatcher-21] [DEBUG] MessageSlicer - member-2-shard-cars-testWriteTransactionWithSingleShard (Leader)_slicer-id-455: Closing 01:50:15,95 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:50:15,95 PM [reinstated-member2-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:50:15,95 PM [reinstated-member2-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 01:50:15,95 PM [reinstated-member2-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 01:50:15,96 PM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:50:15,96 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:50:15,96 PM [main] [WARN] ActorUtils - ShardManager for testWriteTransactionWithSingleShard data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:50:15,96 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:50:15,96 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:50:15,96 PM [main] [WARN] ActorUtils - ShardManager for testWriteTransactionWithSingleShard data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:50:15,97 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:50:15,97 PM [reinstated-member2-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: shutting down 01:50:15,97 PM [reinstated-member2-akka.actor.default-dispatcher-19] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:50:15,97 PM [reinstated-member2-akka.actor.default-dispatcher-19] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2033407108] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2033407108] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:15,98 PM [reinstated-member2-akka.actor.default-dispatcher-19] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://reinstated-member2/user/$a#1566420634] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/$a#1566420634] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:15,99 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2033407108] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2033407108] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:15,100 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:15,100 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:15,101 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:15,112 PM [reinstated-member2-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:15,112 PM [reinstated-member2-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:15,114 PM [reinstated-member2-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:15.115] [reinstated-member2-akka.actor.internal-dispatcher-2] [Cluster(akka://reinstated-member2)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:15,133 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:50:15,139 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [1121557425704984224] 01:50:15,140 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:15,141 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:15,141 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:15,157 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:15,157 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:15,157 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:15,157 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:15,157 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:15,185 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:15,197 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-7531788804127029660] 01:50:15,197 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:15,203 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:15,203 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:15,212 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:15,212 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:15,215 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:15,233 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [5280557721160118596] 01:50:15,234 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:15,235 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:15,235 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:15,241 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:15,241 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:15,248 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:15,249 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:15,249 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:15,251 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,256 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:15,260 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#1829188253] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:15,261 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,265 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: no initial generation override, starting from 0 01:50:15,265 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:50:15,265 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6e8f8cfc to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5dd4d780 01:50:15,265 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@22911720 01:50:15,265 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, 0, 1710165015265, None)) 01:50:15,265 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,265 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Creating local shard: member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,266 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:15,266 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,266 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165015264,0,0)) 01:50:15,266 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5dd4d780 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@ab61639 01:50:15,266 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} started 01:50:15,267 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:15,268 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:15,270 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,271 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@18a162b4 01:50:15,271 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:15,271 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,271 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,271 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,271 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: updated peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#343426780] 01:50:15,271 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, persistent : true 01:50:15,272 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#590168079] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:15,272 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with journal batch size 1 01:50:15,273 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#343426780] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:15,273 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-1:snapshot-read#-701814345] 01:50:15,273 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#-1685728666] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:15,274 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:15,274 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:15,274 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,274 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:15,274 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 01:50:15,274 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Initializing shard [cars] 01:50:15,274 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is initialized 01:50:15,274 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#-1685728666 created and ready for shard:member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,274 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from null to Follower 01:50:15,274 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,274 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from null to Follower 01:50:15,275 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-1:snapshot-read#-701814345] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:15,279 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: no initial generation override, starting from 0 01:50:15,279 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:50:15,279 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@77e06979 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@28fdf8a9 01:50:15,279 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,279 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Creating local shard: member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,280 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:15,280 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,284 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:15,292 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:15,295 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@24a91286 01:50:15,295 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, 0, 1710165015282, None)) 01:50:15,295 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165015281,0,0)) 01:50:15,295 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5a07f5ca 01:50:15,295 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@28fdf8a9 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2cb324e5 01:50:15,295 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} started 01:50:15,296 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:15,296 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,296 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,296 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,296 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: updated peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#598007283] 01:50:15,296 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, persistent : true 01:50:15,296 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#598007283] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:15,297 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#-428775632 created and ready for shard:member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,297 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#-428775632] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:15,297 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-2:snapshot-read#-1337649231] 01:50:15,297 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with journal batch size 1 01:50:15,297 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-2:snapshot-read#-1337649231] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:15,297 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:15,297 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:15,298 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Initializing shard [cars] 01:50:15,298 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is initialized 01:50:15,298 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:15,298 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 01:50:15,298 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,298 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from null to Follower 01:50:15,298 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,298 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from null to Follower 01:50:15,346 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#1829188253] 01:50:15,346 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: findLocalShard : cars 01:50:15,346 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#343426780] 01:50:15,535 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:15,535 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Received ElectionTimeout - switching to Candidate 01:50:15,535 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 01:50:15,535 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,535 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 1 01:50:15,535 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:15,535 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 01:50:15,535 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:15,536 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 01:50:15,536 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Follower to Candidate 01:50:15,536 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Follower to Candidate 01:50:15,825 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:50:15,825 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:15,825 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 2 01:50:15,845 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:16,85 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:50:16,85 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,85 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 3 01:50:16,167 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:16,167 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:16,167 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,167 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:16,167 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,167 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:16,167 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,167 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,167 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,174 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:16,174 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,174 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,174 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:16,174 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,174 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,251 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:16,324 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:50:16,324 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,325 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 4 01:50:16,325 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,327 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:16,327 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Cluster state: [] 01:50:16,327 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:16,327 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:16,327 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=null 01:50:16,327 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:16,327 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,327 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:16,332 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:16,332 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:16,332 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 01:50:16,332 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:16,332 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:16,332 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:50:16,332 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: became leader with frontend state for [] 01:50:16,332 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Candidate to Leader 01:50:16,332 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderPayloadVersion=13] 01:50:16,332 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:16,332 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:16,332 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Candidate to Leader 01:50:16,332 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All Shards are ready - data store testTransactionWithCreateTxFailureDueToNoLeader is ready 01:50:16,333 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:50:16,333 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,333 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderPayloadVersion=13] 01:50:16,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:16,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:16,333 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All Shards are ready - data store testTransactionWithCreateTxFailureDueToNoLeader is ready 01:50:16,333 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader status sync done false 01:50:16,334 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader updated: matchIndex: -1, nextIndex: 0 01:50:16,334 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:16,352 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 01:50:16,352 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#590168079] 01:50:16,352 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:16,352 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 Queuing TransactionOperation 01:50:16,352 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, primaryVersion=13] 01:50:16,352 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, primaryVersion=13] 01:50:16,353 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] for shard cars 01:50:16,353 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 01:50:16,353 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 01:50:16,354 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, disabling transaction tracking 01:50:16,354 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 01:50:16,354 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 01:50:16,354 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:16,354 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:16,354 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,355 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}]]] 01:50:16,355 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Creating transaction : member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 01:50:16,355 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_13#-1945702363] 01:50:16,356 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_13#-1945702363] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:16,356 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 01:50:16,363 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_13#-1945702363, transactionId=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0, version=13] 01:50:16,363 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:16,363 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_13#-1945702363)] 01:50:16,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}]]] 01:50:16,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Number of entries to be appended = 1 01:50:16,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:16,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74} 01:50:16,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 01:50:16,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Log size is now 1 01:50:16,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:16,364 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 01:50:16,364 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader status sync done true 01:50:16,364 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader updated: matchIndex: 0, nextIndex: 1 01:50:16,364 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:16,364 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Setting last applied to 0 01:50:16,364 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Moving last applied index from -1 to 0 01:50:16,364 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74} 01:50:16,364 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:50:16,364 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} does not match any client, pre-disabling client. 01:50:16,364 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, nanosAgo=10191351} is already disabled 01:50:16,364 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:16,364 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Persisting ApplyJournalEntries with index=0 01:50:16,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Commit index set to 0 01:50:16,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:16,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Setting last applied to 0 01:50:16,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Moving last applied index from -1 to 0 01:50:16,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74} 01:50:16,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:50:16,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} does not match any client, pre-disabling client. 01:50:16,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Persisting ApplyJournalEntries with index=0 01:50:16,366 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 ReadData operation succeeded 01:50:16,366 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:50:16,367 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,367 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader)_slicer-id-460: Closing 01:50:16,367 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Aborting 0 pending queued transactions 01:50:16,367 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,367 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:16,367 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:16,371 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:16,372 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:16,372 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:16,372 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:16,374 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:16,388 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 01:50:16,486 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,487 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionWithCreateTxFailureDueToNoLeader commit rateLimit adjusted to 162.12298098119095 01:50:16,487 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 01:50:16,487 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:16,487 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:16,487 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,487 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 01:50:16,487 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 01:50:16,487 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 01:50:16,487 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,487 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Queuing TransactionOperation 01:50:16,488 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 01:50:16,488 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : true 01:50:16,488 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 01:50:16,488 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Readying 1 components for commit 01:50:16,488 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Readying transaction for shard cars 01:50:16,488 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Queuing TransactionOperation 01:50:16,488 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 canCommit 01:50:16,515 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)]] after [10 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 01:50:16,545 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 01:50:16,547 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 01:50:16,547 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 01:50:16,575 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)]] after [10 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 01:50:16,605 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 01:50:16,605 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 01:50:16,605 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 01:50:16,636 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)]] after [10 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 01:50:16,665 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 01:50:16,665 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 01:50:16,665 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 01:50:16,695 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)]] after [10 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 01:50:16,706 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 01:50:16,706 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 01:50:16,706 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 01:50:16,706 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Received ElectionTimeout - switching to Candidate 01:50:16,706 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 01:50:16,706 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,706 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 5 01:50:16,706 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,707 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:50:16,707 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 01:50:16,707 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:16,707 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 01:50:16,707 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:50:16,707 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Follower to Candidate 01:50:16,707 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=null, leaderPayloadVersion=-1] 01:50:16,707 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:16,707 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:16,707 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Follower to Candidate 01:50:16,725 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 01:50:16,725 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#590168079] 01:50:16,725 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:16,725 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Candidate, leaderId=null, activeMember=true] 01:50:16,845 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:50:16,845 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=6, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,845 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 6 01:50:16,845 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,945 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received ShardNotInitializedTimeout message for shard cars 01:50:16,945 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Returning NoShardLeaderException for shard cars 01:50:16,945 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: Find primary for shard cars failed org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:16,946 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Creating NoOpTransaction because of error org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:16,946 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] NoOpTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:16,946 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] NoOpTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 directCommit called, failure org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:16,947 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:50:16,947 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithCreateTxFailureDueToNoLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:50:16,947 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:50:16,947 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:50:16,947 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithCreateTxFailureDueToNoLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:50:16,948 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:50:16,948 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:50:16,949 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,949 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Aborting 0 pending queued transactions 01:50:16,949 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:50:16,950 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:16,950 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:17,382 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [1121557425704984224] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:17,971 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:17,971 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-7531788804127029660] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:17,972 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:17.973] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:17,986 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:17,986 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:17,988 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:17.989] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:18,30 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:18,39 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-1169234816577892225] 01:50:18,39 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:18,45 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:18,45 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:18,48 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:18,48 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:18,55 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:18,55 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:18,55 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:18,67 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:18,77 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-8644148385415062133] 01:50:18,77 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:18,90 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:18,90 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:18,98 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:18,98 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:18,130 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:18,153 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [396666191594875750] 01:50:18,154 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:18,156 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:18,156 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:18,157 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:18,166 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:18,166 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:18,167 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:18,169 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithMultipleShards 01:50:18,174 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:18,174 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:18,180 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 01:50:18,181 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#954567791] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,186 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: no initial generation override, starting from 0 01:50:18,186 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:50:18,186 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@77b0a6b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@213660f9 01:50:18,186 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@74485571 01:50:18,186 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, 0, 1710165018186, None)) 01:50:18,186 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithMultipleShards 01:50:18,186 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:18,186 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165018185,0,0)) 01:50:18,186 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@213660f9 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@246b0a3f 01:50:18,187 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:18,187 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} started 01:50:18,187 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:18,187 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:18,187 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:18,187 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1a4e1d69 01:50:18,187 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:18,187 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:18,187 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:18,187 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:18,187 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:18,187 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-2034081102] 01:50:18,187 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:18,187 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:18,187 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:18,187 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards on shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#2017149893] 01:50:18,188 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, persistent : true 01:50:18,188 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-2034081102] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,188 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-1622401429 created and ready for shard:member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:18,188 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-1622401429] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,188 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:18,188 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-1563123620] 01:50:18,189 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-people-testChainedTransactionFailureWithMultipleShards, persistent : true 01:50:18,190 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-people-testChainedTransactionFailureWithMultipleShards with journal batch size 1 01:50:18,190 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithMultipleShards 01:50:18,190 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-1271506679 created and ready for shard:member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:18,190 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-1492700577] 01:50:18,191 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:18,191 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:18,206 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:50:18,206 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:50:18,206 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [people] 01:50:18,206 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is initialized 01:50:18,206 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 01:50:18,206 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 01:50:18,206 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from null to Follower 01:50:18,206 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-1563123620] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,206 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#2017149893] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,206 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-1271506679] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,206 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-1492700577] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,207 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 01:50:18,207 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#737351197] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,209 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with journal batch size 1 01:50:18,210 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:18,210 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:18,210 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:50:18,210 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:50:18,210 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [cars] 01:50:18,210 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is initialized 01:50:18,210 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 01:50:18,210 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 01:50:18,210 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from null to Follower 01:50:18,220 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: no initial generation override, starting from 0 01:50:18,220 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:50:18,220 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6f70e73b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@77b5ec77 01:50:18,220 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithMultipleShards 01:50:18,220 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@32b76ce5 01:50:18,220 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:18,220 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, 0, 1710165018220, None)) 01:50:18,220 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:18,221 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165018219,0,0)) 01:50:18,221 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@77b5ec77 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5bc10c73 01:50:18,221 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} started 01:50:18,222 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:18,223 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:18,223 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:18,223 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@32ff56f2 01:50:18,223 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:18,223 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:18,226 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:18,226 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:18,226 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#-1257518280] 01:50:18,226 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:18,226 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:18,226 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:18,226 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards on shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards#440300472] 01:50:18,227 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, persistent : true 01:50:18,227 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-people-testChainedTransactionFailureWithMultipleShards, persistent : true 01:50:18,227 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with journal batch size 1 01:50:18,228 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-people-testChainedTransactionFailureWithMultipleShards with journal batch size 1 01:50:18,228 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-249150550 created and ready for shard:member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:18,228 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#-1830715894] 01:50:18,232 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#-1257518280] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,240 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:18,243 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:18,246 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards#440300472] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,246 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-249150550] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,246 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#-1830715894] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,246 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:18,246 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:18,247 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [people] 01:50:18,247 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards is initialized 01:50:18,247 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:50:18,247 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:50:18,247 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:18,247 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-554661860 created and ready for shard:member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:18,247 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-554661860] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,247 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 01:50:18,247 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 01:50:18,247 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-2-shard-people-testChainedTransactionFailureWithMultipleShards from null to Follower 01:50:18,248 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#965989739] 01:50:18,248 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#965989739] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:18,248 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:18,248 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:18,249 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [cars] 01:50:18,249 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards is initialized 01:50:18,249 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:50:18,249 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:50:18,249 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 01:50:18,249 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:18,249 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 01:50:18,249 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards from null to Follower 01:50:18,271 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#954567791] 01:50:18,271 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: findLocalShard : cars 01:50:18,271 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-2034081102] 01:50:18,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:18,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:50:18,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testChainedTransactionFailureWithMultipleShards] 01:50:18,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:18,431 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 1 01:50:18,432 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:18,432 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Closing 01:50:18,432 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:50:18,432 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:50:18,432 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from Follower to Candidate 01:50:18,432 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from Follower to Candidate 01:50:18,501 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:18,501 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:50:18,501 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testChainedTransactionFailureWithMultipleShards] 01:50:18,501 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:18,501 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 1 01:50:18,501 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:18,501 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Closing 01:50:18,501 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:50:18,501 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:50:18,501 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from Follower to Candidate 01:50:18,501 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from Follower to Candidate 01:50:18,741 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:50:18,741 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:18,741 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 2 01:50:18,771 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:50:18,771 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:18,771 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 2 01:50:18,793 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:18,854 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:19,21 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:50:19,21 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:19,21 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 3 01:50:19,41 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:50:19,41 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,41 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 3 01:50:19,62 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:19,62 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:19,63 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:19,63 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,63 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,63 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:19,63 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:19,63 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:19,63 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,63 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,63 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:19,63 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:19,63 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,63 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:19,75 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:19,75 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,75 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,75 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:19,75 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:19,75 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:19,75 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,75 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,75 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:19,75 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:19,183 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:19,311 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:50:19,311 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:19,311 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 4 01:50:19,311 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:19,313 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:19,313 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Cluster state: [] 01:50:19,313 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:19,314 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:19,314 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=null 01:50:19,314 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:19,314 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:19,314 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:19,315 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:19,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:19,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testChainedTransactionFailureWithMultipleShards] 01:50:19,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:19,316 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:19,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:50:19,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: became leader with frontend state for [] 01:50:19,316 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from Candidate to Leader 01:50:19,316 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 01:50:19,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:19,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:19,316 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from Candidate to Leader 01:50:19,319 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Closing 01:50:19,319 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testChainedTransactionFailureWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:19,319 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 01:50:19,319 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:19,319 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:19,319 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-people-testChainedTransactionFailureWithMultipleShards status sync done false 01:50:19,320 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:50:19,320 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:19,331 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:50:19,331 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,331 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 4 01:50:19,331 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,333 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:19,333 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Cluster state: [] 01:50:19,333 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:19,333 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:19,333 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=null 01:50:19,333 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:19,333 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,333 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:19,334 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:19,334 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:19,334 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testChainedTransactionFailureWithMultipleShards] 01:50:19,335 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:19,335 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:19,335 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:50:19,335 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: became leader with frontend state for [] 01:50:19,335 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from Candidate to Leader 01:50:19,335 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 01:50:19,335 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:19,335 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:19,335 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from Candidate to Leader 01:50:19,335 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All Shards are ready - data store testChainedTransactionFailureWithMultipleShards is ready 01:50:19,336 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Closing 01:50:19,336 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testChainedTransactionFailureWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,336 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards status sync done false 01:50:19,336 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 01:50:19,336 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:19,336 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:19,336 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All Shards are ready - data store testChainedTransactionFailureWithMultipleShards is ready 01:50:19,337 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:50:19,337 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:19,428 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#954567791] 01:50:19,428 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: findLocalShard : people 01:50:19,428 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#2017149893] 01:50:19,429 PM [main] [DEBUG] AbstractDOMBroker - Transaction chain 0 created with listener Mock for DOMTransactionChainListener, hashCode: 579872327, backing store chains {CONFIGURATION=org.opendaylight.controller.cluster.datastore.TransactionChainProxy@5b65e2d0} 01:50:19,430 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testChainedTransactionFailureWithMultipleShards commit rateLimit adjusted to 162.12298098119095 01:50:19,430 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 01:50:19,430 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#737351197] 01:50:19,431 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:50:19,431 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#737351197] 01:50:19,431 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:50:19,431 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:50:19,431 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 01:50:19,431 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:19,432 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 01:50:19,432 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 01:50:19,432 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Readying 2 components for commit 01:50:19,432 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 01:50:19,432 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:50:19,432 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Readying transaction for shard people 01:50:19,432 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:50:19,432 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards)] for shard people 01:50:19,432 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards)] found - creating WRITE_ONLY transaction context 01:50:19,432 PM [main] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 is submitted for execution. 01:50:19,432 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 canCommit 01:50:19,432 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 01:50:19,432 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 = [ 1 ] 01:50:19,432 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction called 01:50:19,432 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 sending 1 batched modifications, ready: true 01:50:19,433 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards)] 01:50:19,433 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 01:50:19,433 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards)] for shard cars 01:50:19,433 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards)] found - creating WRITE_ONLY transaction context 01:50:19,433 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:19,433 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 = [ 1 ] 01:50:19,433 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction called 01:50:19,433 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 sending 1 batched modifications, ready: true 01:50:19,433 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards)] 01:50:19,436 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-people-testChainedTransactionFailureWithMultipleShards status sync done true 01:50:19,439 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, disabling transaction tracking 01:50:19,439 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, disabling transaction tracking 01:50:19,439 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 01:50:19,439 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 01:50:19,439 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 01:50:19,439 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 01:50:19,439 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:19,439 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:19,439 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:19,439 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:19,439 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,439 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:19,439 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}]]] 01:50:19,439 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}]]] 01:50:19,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 01:50:19,440 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 01:50:19,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 01:50:19,440 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 01:50:19,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:50:19,440 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:50:19,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} 01:50:19,440 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} 01:50:19,440 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@326afa67 01:50:19,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@684a24ae 01:50:19,440 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 01:50:19,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 01:50:19,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Readying Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 of 1 operations, client version 13 01:50:19,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: readying transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 01:50:19,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 ready 01:50:19,440 PM [cluster-test-shard-dispatcher-22] [ERROR] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Error handling BatchedModifications for Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:19,441 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 01:50:19,441 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 01:50:19,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE}}}]} 01:50:19,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,453 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}]]] 01:50:19,453 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 01:50:19,453 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:19,453 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 01:50:19,453 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 01:50:19,453 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 1 01:50:19,453 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:19,453 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}]]] 01:50:19,453 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 01:50:19,453 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:19,453 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 01:50:19,453 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 01:50:19,453 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 1 01:50:19,453 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:19,453 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: a cohort Future failed org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:19,453 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during phase CAN_COMMIT, starting Abort org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:19,454 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DOMBrokerTransactionChain - Transaction chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@324fcd90 failed. TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema.]]} at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1119) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) at org.opendaylight.controller.cluster.datastore.ThreePhaseCommitCohortProxy$4.onFailure(ThreePhaseCommitCohortProxy.java:140) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1119) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) at org.opendaylight.controller.cluster.datastore.ThreePhaseCommitCohortProxy$3.onComplete(ThreePhaseCommitCohortProxy.java:102) at org.opendaylight.controller.cluster.datastore.ThreePhaseCommitCohortProxy$3.onComplete(ThreePhaseCommitCohortProxy.java:95) at akka.dispatch.OnComplete.internal(Future.scala:299) at akka.dispatch.OnComplete.internal(Future.scala:297) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:19,454 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 abort 01:50:19,454 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: a abort cohort path Future failed org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:19,454 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 aborted successfully 01:50:19,454 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 modifications failed org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:19,454 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction 01:50:19,455 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:50:19,455 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 01:50:19,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 01:50:19,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 01:50:19,455 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards status sync done true 01:50:19,455 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 01:50:19,456 PM [main] [DEBUG] DOMBrokerTransactionChain - Chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@324fcd90 is no longer running 01:50:19,456 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:50:19,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:19,456 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards - waiting on 2 shards 01:50:19,456 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onShutDown 01:50:19,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onShutDown 01:50:19,456 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Initiating leader transfer 01:50:19,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Initiating leader transfer 01:50:19,457 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4175cebd 01:50:19,457 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@f251fac 01:50:19,457 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@f251fac 01:50:19,457 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Attempting to transfer leadership 01:50:19,457 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:19,457 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,457 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:50:19,457 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:19,457 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:19,457 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:50:19,457 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:19,457 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:19,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:50:19,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:19,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 0 01:50:19,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 01:50:19,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 01:50:19,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:50:19,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:50:19,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, nanosAgo=19222602} is already disabled 01:50:19,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:19,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}]]] 01:50:19,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, matchIndex: 0, lastIndex: 1, isVoting: true 01:50:19,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:50:19,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:50:19,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:19,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 0 01:50:19,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 01:50:19,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 01:50:19,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:50:19,458 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:50:19,459 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, nanosAgo=19750165} is already disabled 01:50:19,459 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:19,459 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:19,459 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}]]] 01:50:19,459 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:50:19,460 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards] 01:50:19,460 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:50:19,460 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:19,460 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:19,460 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards] 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}]]] 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 2 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 0 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 0 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}]]] 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 2 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 0 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 0 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:50:19,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:50:19,461 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 01:50:19,461 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:50:19,461 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 01:50:19,461 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:50:19,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:50:19,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:19,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 1 01:50:19,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 01:50:19,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 01:50:19,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} has no callback 01:50:19,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:50:19,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:19,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:50:19,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:50:19,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:19,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Leader transfer complete 01:50:19,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:50:19,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:50:19,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:19,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 1 01:50:19,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 01:50:19,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 01:50:19,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} has no callback 01:50:19,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:50:19,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:19,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:50:19,465 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 1 01:50:19,465 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:19,465 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 1 01:50:19,465 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 01:50:19,465 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 01:50:19,466 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:50:19,466 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:50:19,466 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 1 01:50:19,466 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:19,466 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 1 01:50:19,466 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 01:50:19,466 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 01:50:19,466 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:50:19,466 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:50:19,671 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Attempting to transfer leadership 01:50:19,671 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:19,674 PM [cluster-test-shard-dispatcher-31] [WARN] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 206, lastApplied : 1, commitIndex : 1 01:50:19,674 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testChainedTransactionFailureWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:50:19,674 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:50:19,674 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:19,674 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Leader transfer complete 01:50:19,871 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All shards shutdown - sending PoisonPill to self 01:50:19,871 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-2034081102]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:19,871 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,871 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader)_slicer-id-468: Closing 01:50:19,871 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:19,871 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader)_slicer-id-467: Closing 01:50:19,872 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 01:50:19,872 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Aborting 1 pending queued transactions 01:50:19,872 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 01:50:19,872 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 01:50:19,872 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:50:19,873 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 01:50:19,873 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#954567791] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#954567791] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:19,873 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#954567791] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#954567791] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:19,873 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: shutting down 01:50:19,873 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 01:50:19,875 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:19,875 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:19,875 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#954567791] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#954567791] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:19,881 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:19,881 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:19,881 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:19,881 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:19,887 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:19,902 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 01:50:19,903 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:50:19,903 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Closing 01:50:19,903 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 01:50:19,903 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:50:19,903 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Closing 01:50:19,904 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 01:50:19,904 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 01:50:19,904 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:19,904 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:20,893 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-1169234816577892225] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:20,922 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-8644148385415062133] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:20,926 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:20,926 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:20,928 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:20.930] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:20,948 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:20,948 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:20,950 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:20.951] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:20,972 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:20,979 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4675357995021967139] 01:50:20,979 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:20,980 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:20,980 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:20,981 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:20,981 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:21,2 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 closeTransaction called 01:50:21,2 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_10#-120739951)] 01:50:21,2 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:21,2 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:21,2 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:21,2 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 closeTransaction called 01:50:21,2 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_12#2077972844)] 01:50:21,2 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 closeTransaction called 01:50:21,2 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_13#-1945702363)] 01:50:21,2 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 closeTransaction called 01:50:21,2 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_11#1506371316)] 01:50:21,3 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:21,8 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-5586799686954147825] 01:50:21,8 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:21,9 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:21,9 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:21,15 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:21,15 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:21,39 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:21,52 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [1629322219040464183] 01:50:21,52 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:21,52 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:21,55 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:21,56 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:21,56 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:21,57 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:21,57 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:21,66 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:21,67 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:21,68 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 01:50:21,72 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1020793390] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:21,75 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 01:50:21,81 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 01:50:21,81 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:21,81 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 01:50:21,81 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1950e5e8 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@fc908a 01:50:21,81 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:21,81 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3f2989eb 01:50:21,81 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-1-shard-people-testLeadershipTransferOnShutdown 01:50:21,82 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:21,82 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:21,82 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:21,82 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1710165021082, None)) 01:50:21,82 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165021081,0,0)) 01:50:21,82 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@fc908a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7e3d68c5 01:50:21,82 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 01:50:21,83 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:21,84 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:21,86 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 01:50:21,87 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@539254f3 01:50:21,87 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:21,87 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:21,87 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:21,87 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:21,87 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-1-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-280617298] 01:50:21,87 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testLeadershipTransferOnShutdown 01:50:21,87 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:21,87 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:21,87 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-1-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1765595066] 01:50:21,87 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Shard created : member-1-shard-cars-testLeadershipTransferOnShutdown, persistent : true 01:50:21,87 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-people-testLeadershipTransferOnShutdown, persistent : true 01:50:21,89 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Starting recovery for member-1-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 01:50:21,89 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-280617298] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:21,89 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1765595066] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:21,89 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#-2132537549 created and ready for shard:member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:21,90 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#-2132537549] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:21,90 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:21,90 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:21,90 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:21,90 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:50:21,90 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 01:50:21,91 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 01:50:21,91 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-testLeadershipTransferOnShutdown is initialized 01:50:21,92 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-564530329] 01:50:21,92 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-564530329] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:21,94 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 01:50:21,95 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-notifier#-1412114136 created and ready for shard:member-1-shard-people-testLeadershipTransferOnShutdown 01:50:21,95 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-notifier#-1412114136] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:21,95 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#1886004614] 01:50:21,95 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:50:21,95 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#1886004614] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:21,95 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from null to Follower 01:50:21,95 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 01:50:21,96 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:21,96 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:21,96 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:21,96 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:50:21,96 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 01:50:21,96 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-people-testLeadershipTransferOnShutdown is initialized 01:50:21,96 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1204393993] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:21,97 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 01:50:21,97 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:50:21,97 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from null to Follower 01:50:21,100 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 01:50:21,100 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:21,100 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2d441ba0 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4297ccc5 01:50:21,100 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@46a8404f 01:50:21,100 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1710165021100, None)) 01:50:21,101 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165021099,0,0)) 01:50:21,101 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4297ccc5 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@30e8715c 01:50:21,101 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 01:50:21,101 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:21,101 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 01:50:21,101 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-2-shard-people-testLeadershipTransferOnShutdown 01:50:21,101 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:21,101 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:21,101 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:21,101 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2167a61f 01:50:21,102 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:21,102 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:21,102 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:21,102 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:21,102 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-2-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-1374940027] 01:50:21,102 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testLeadershipTransferOnShutdown 01:50:21,102 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testLeadershipTransferOnShutdown, persistent : true 01:50:21,102 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:21,103 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:50:21,103 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-2-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-2128892497] 01:50:21,103 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-people-testLeadershipTransferOnShutdown, persistent : true 01:50:21,103 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 01:50:21,103 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 01:50:21,104 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-1374940027] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:21,104 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-2128892497] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:21,104 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-notifier#-1834566471 created and ready for shard:member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:21,104 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-notifier#-1834566471] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:21,105 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-notifier#-1869122603 created and ready for shard:member-2-shard-people-testLeadershipTransferOnShutdown 01:50:21,105 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-notifier#-1869122603] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:21,105 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-894320228] 01:50:21,105 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-894320228] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:21,107 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:21,107 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:21,107 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:21,107 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:50:21,107 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 01:50:21,107 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is initialized 01:50:21,107 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:21,107 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 01:50:21,107 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:50:21,107 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from null to Follower 01:50:21,107 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-470615465] 01:50:21,107 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-470615465] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:21,109 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:21,109 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:21,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 01:50:21,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized 01:50:21,109 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:21,109 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:50:21,109 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:21,109 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 01:50:21,109 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:50:21,109 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from null to Follower 01:50:21,115 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:21,122 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:21,151 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1020793390] 01:50:21,152 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 01:50:21,152 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-280617298] 01:50:21,408 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:21,408 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 01:50:21,408 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:21,408 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 01:50:21,408 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 01:50:21,408 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 01:50:21,408 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-3-shard-cars-testLeadershipTransferOnShutdown] 01:50:21,408 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 01:50:21,408 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:21,408 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:21,408 PM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 01:50:21,408 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:21,408 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:50:21,408 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 01:50:21,408 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:50:21,408 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:21,408 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 01:50:21,408 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 01:50:21,408 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:21,408 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:50:21,408 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:50:21,408 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 01:50:21,648 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:50:21,648 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=2, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 01:50:21,648 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 2 01:50:21,658 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:50:21,658 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=2, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:21,658 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 2 01:50:21,918 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:50:21,918 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:21,918 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 3 01:50:21,938 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:50:21,938 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 01:50:21,938 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 3 01:50:21,999 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:21,999 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:21,999 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:21,999 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:21,999 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:22,0 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:50:22,0 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:22,0 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-3-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,0 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:22,0 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:22,0 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:22,0 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Peer address for peer member-3-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:50:22,0 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:50:22,0 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:22,0 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:22,0 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:50:22,11 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:22,11 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,11 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:22,11 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:50:22,11 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:50:22,11 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-3-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,11 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:22,11 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:22,11 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-3-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:50:22,11 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:22,11 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:50:22,11 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:50:22,70 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:22,162 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 01:50:22,162 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 01:50:22,162 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 01:50:22,162 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 01:50:22,162 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 01:50:22,162 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=1, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-people-testLeadershipTransferOnShutdown 01:50:22,162 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=1, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-testLeadershipTransferOnShutdown 01:50:22,163 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:22,163 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 01:50:22,163 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:22,163 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:50:22,163 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:50:22,163 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 01:50:22,164 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVote [term=1, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1], my term is 3 01:50:22,164 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): In requestVote: RequestVote [term=1, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: member-1-shard-people-testLeadershipTransferOnShutdown, lastIndex: -1, lastTerm: -1 01:50:22,164 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): requestVote returning: RequestVoteReply [term=3, voteGranted=false] 01:50:22,164 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-2128892497] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:22,165 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=false], my term is 1 01:50:22,165 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Term 3 in "RequestVoteReply [term=3, voteGranted=false]" message is greater than Candidate's term 1 - switching to Follower 01:50:22,165 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=null 01:50:22,165 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Follower, election term: 3 01:50:22,165 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:22,165 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:50:22,166 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Follower 01:50:22,166 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Candidate to Follower 01:50:22,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 01:50:22,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 01:50:22,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 01:50:22,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:22,192 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 01:50:22,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=1, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=1, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:22,192 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:22,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 01:50:22,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:22,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:50:22,192 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:50:22,192 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 01:50:22,194 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVote [term=1, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1], my term is 3 01:50:22,194 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): In requestVote: RequestVote [term=1, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: member-1-shard-cars-testLeadershipTransferOnShutdown, lastIndex: -1, lastTerm: -1 01:50:22,194 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): requestVote returning: RequestVoteReply [term=3, voteGranted=false] 01:50:22,195 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-1374940027] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:22,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=false], my term is 1 01:50:22,196 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Term 3 in "RequestVoteReply [term=3, voteGranted=false]" message is greater than Candidate's term 1 - switching to Follower 01:50:22,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=null 01:50:22,196 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Follower, election term: 3 01:50:22,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:22,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:50:22,196 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Follower 01:50:22,196 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Candidate to Follower 01:50:22,198 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:50:22,198 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:22,198 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 4 01:50:22,198 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:22,198 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:22,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 01:50:22,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:22,199 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 3 - updating term 01:50:22,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 01:50:22,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:22,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:22,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:22,200 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-280617298] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:22,201 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:22,201 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:22,201 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-3-shard-cars-testLeadershipTransferOnShutdown] 01:50:22,201 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:22,201 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:22,201 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:22,201 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 01:50:22,202 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [] 01:50:22,202 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:50:22,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:22,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:22,202 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 01:50:22,202 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 01:50:22,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 01:50:22,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:22,203 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done false 01:50:22,203 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:50:22,203 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:22,203 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:22,203 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-280617298] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:22,203 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 01:50:22,203 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:22,256 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1020793390] 01:50:22,256 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : people 01:50:22,257 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1765595066] 01:50:22,258 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:50:22,258 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 01:50:22,258 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 4 01:50:22,258 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testLeadershipTransferOnShutdown 01:50:22,259 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-testLeadershipTransferOnShutdown 01:50:22,260 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:22,260 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 01:50:22,260 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:22,260 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 3 - updating term 01:50:22,260 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 01:50:22,260 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:22,260 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 01:50:22,260 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:22,260 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1765595066] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:22,262 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:22,262 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:22,262 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 01:50:22,262 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:22,262 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:22,262 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:22,262 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 01:50:22,262 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: became leader with frontend state for [] 01:50:22,262 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 01:50:22,262 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:50:22,262 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:22,262 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:22,262 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Candidate to Leader 01:50:22,263 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:50:22,267 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1765595066] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:22,267 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown: Closing 01:50:22,267 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:22,267 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-people-testLeadershipTransferOnShutdown status sync done false 01:50:22,267 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:50:22,267 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:22,267 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:22,268 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:50:22,271 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 01:50:22,271 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:22,307 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:22,309 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:22,312 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 01:50:22,316 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#2036024289] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:22,316 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 01:50:22,322 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-280617298] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:22,322 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done true 01:50:22,324 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 01:50:22,324 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 01:50:22,324 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:22,324 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,324 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@fcdbb20 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@557e220 01:50:22,324 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@45c69050 01:50:22,324 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-3-shard-people-testLeadershipTransferOnShutdown 01:50:22,324 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1710165022324, None)) 01:50:22,324 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:22,324 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,324 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:50:22,324 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:22,325 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:22,325 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:22,325 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:22,325 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:22,325 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:50:22,326 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165022323,0,0)) 01:50:22,326 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@557e220 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@21a490a5 01:50:22,326 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 01:50:22,326 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@38ff808 01:50:22,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:22,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,327 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:22,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,327 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testLeadershipTransferOnShutdown commit rateLimit adjusted to 162.12298098119095 01:50:22,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1797172923] 01:50:22,327 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:22,327 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-2-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1797172923] 01:50:22,327 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,327 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1204393993] 01:50:22,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-3-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1797172923] 01:50:22,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-people-testLeadershipTransferOnShutdown 01:50:22,327 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 01:50:22,327 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 01:50:22,328 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1204393993] 01:50:22,328 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 01:50:22,328 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Readying 2 components for commit 01:50:22,328 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Readying transaction for shard cars 01:50:22,328 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:22,328 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 01:50:22,328 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:50:22,328 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Readying transaction for shard people 01:50:22,328 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 01:50:22,328 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#-192771579] 01:50:22,328 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:50:22,328 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:50:22,328 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-2-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#-192771579] 01:50:22,328 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:50:22,328 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:50:22,328 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-3-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#-192771579] 01:50:22,328 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-3-shard-cars-testLeadershipTransferOnShutdown, persistent : true 01:50:22,328 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-3-shard-people-testLeadershipTransferOnShutdown, persistent : true 01:50:22,328 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-3-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 01:50:22,329 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1020793390] 01:50:22,329 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-notifier#705238914 created and ready for shard:member-3-shard-people-testLeadershipTransferOnShutdown 01:50:22,329 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#1520826855] 01:50:22,329 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-3-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 01:50:22,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:22,329 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1797172923] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:22,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 01:50:22,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:50:22,329 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 01:50:22,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown, primaryVersion=13] 01:50:22,329 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-notifier#-1435485029 created and ready for shard:member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,329 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] for shard cars 01:50:22,329 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] found - creating WRITE_ONLY transaction context 01:50:22,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 01:50:22,329 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:22,329 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:50:22,329 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 = [ 2 ] 01:50:22,329 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 readyTransaction called 01:50:22,329 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 sending 2 batched modifications, ready: true 01:50:22,329 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown, primaryVersion=13] 01:50:22,329 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] 01:50:22,330 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown)] for shard people 01:50:22,330 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown)] found - creating WRITE_ONLY transaction context 01:50:22,330 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 01:50:22,330 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 = [ 1 ] 01:50:22,330 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 readyTransaction called 01:50:22,330 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 sending 1 batched modifications, ready: true 01:50:22,330 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown)] 01:50:22,330 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#-73642580] 01:50:22,330 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-280617298] 01:50:22,330 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-280617298] 01:50:22,331 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#-192771579] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:22,331 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:22,331 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:22,331 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:22,331 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:50:22,331 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:22,331 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:50:22,331 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 01:50:22,331 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-3-shard-people-testLeadershipTransferOnShutdown is initialized 01:50:22,331 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 01:50:22,331 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:50:22,331 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-people-testLeadershipTransferOnShutdown from null to Follower 01:50:22,332 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, disabling transaction tracking 01:50:22,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:50:22,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:50:22,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:22,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:22,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:22,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 01:50:22,333 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, disabling transaction tracking 01:50:22,333 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:50:22,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:22,333 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:50:22,333 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:22,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying 2 batched modifications for Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,333 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:22,333 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 01:50:22,333 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 01:50:22,333 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:22,333 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying 1 batched modifications for Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Readying Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 of 2 operations, client version 13 01:50:22,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 ready 01:50:22,334 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Readying Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 of 1 operations, client version 13 01:50:22,334 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,334 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:50:22,334 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 ready 01:50:22,334 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 01:50:22,334 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:50:22,334 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:22,334 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:50:22,334 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:50:22,335 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 01:50:22,335 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 readyTransaction 01:50:22,335 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:50:22,335 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:22,335 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:50:22,335 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:50:22,335 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:50:22,335 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 1 01:50:22,335 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 1 01:50:22,335 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:22,335 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:22,335 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:50:22,335 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 01:50:22,335 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 01:50:22,335 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-3-shard-people-testLeadershipTransferOnShutdown: Closing 01:50:22,335 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:50:22,335 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:50:22,335 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-people-testLeadershipTransferOnShutdown status sync done true 01:50:22,335 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:50:22,336 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 readyTransaction 01:50:22,336 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:50:22,336 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:50:22,336 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:22,336 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:22,336 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-people-testLeadershipTransferOnShutdown status sync done false 01:50:22,336 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:22,337 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:22,337 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:22,337 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:50:22,337 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 01:50:22,337 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 01:50:22,337 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:22,337 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Setting last applied to 0 01:50:22,337 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:50:22,337 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:50:22,337 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:22,337 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 01:50:22,337 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendMetadata - member-1-shard-people-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 01:50:22,337 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 01:50:22,337 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:22,337 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 01:50:22,337 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=4047573} is already disabled 01:50:22,337 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 0 01:50:22,337 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:50:22,337 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:50:22,337 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:22,337 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-cars-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 01:50:22,337 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:22,337 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=4435703} is already disabled 01:50:22,337 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:22,337 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 01:50:22,337 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 01:50:22,337 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:22,337 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 01:50:22,337 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:50:22,337 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:22,337 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 01:50:22,337 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:50:22,337 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 01:50:22,337 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-3-shard-cars-testLeadershipTransferOnShutdown is initialized 01:50:22,338 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 01:50:22,338 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:22,338 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,338 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 01:50:22,338 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 01:50:22,338 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:22,338 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 01:50:22,338 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:50:22,338 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:50:22,338 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:22,338 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-people-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 01:50:22,338 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 01:50:22,338 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:50:22,338 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:22,338 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 01:50:22,338 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:50:22,338 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:50:22,338 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:22,338 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 01:50:22,338 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:50:22,339 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:22,339 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:22,339 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done false 01:50:22,339 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:50:22,339 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 01:50:22,339 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:50:22,339 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:22,339 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:50:22,339 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:50:22,339 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:50:22,339 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:22,339 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 1 01:50:22,339 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:22,339 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 01:50:22,339 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from null to Follower 01:50:22,339 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:22,339 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:22,339 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 01:50:22,339 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:50:22,339 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:50:22,339 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:50:22,339 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:22,339 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-3-shard-people-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 01:50:22,339 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:50:22,340 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:50:22,340 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-people-testLeadershipTransferOnShutdown status sync done true 01:50:22,340 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 01:50:22,340 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:50:22,340 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:22,340 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:50:22,340 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:50:22,340 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 1 01:50:22,340 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 01:50:22,340 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:22,340 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:22,340 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 01:50:22,340 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:50:22,340 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 01:50:22,340 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:22,340 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-3-shard-cars-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 01:50:22,341 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 01:50:22,341 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:50:22,341 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 01:50:22,341 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:22,341 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done true 01:50:22,341 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 01:50:22,341 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:22,380 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-280617298] 01:50:22,381 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:50:22,381 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:22,381 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:22,381 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] found - creating WRITE_ONLY transaction context 01:50:22,381 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 01:50:22,381 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 Readying 1 components for commit 01:50:22,381 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 Readying transaction for shard cars 01:50:22,381 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 directCommit called 01:50:22,381 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 sending 1 batched modifications, ready: true 01:50:22,381 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] 01:50:22,382 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1020793390] 01:50:22,382 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 01:50:22,383 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-280617298] 01:50:22,383 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-280617298] 01:50:22,384 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying 1 batched modifications for Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:50:22,384 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Readying Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 of 1 operations, client version 13 01:50:22,384 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:50:22,384 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 ready 01:50:22,384 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 is scheduled for canCommit step 01:50:22,433 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-280617298] 01:50:22,434 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testLeadershipTransferOnShutdown 01:50:22,434 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1020793390] 01:50:22,434 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:22,435 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-people-testLeadershipTransferOnShutdown 01:50:22,435 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 01:50:22,435 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 01:50:22,435 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 01:50:22,435 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-280617298] 01:50:22,435 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 canCommit 01:50:22,435 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onShutDown 01:50:22,435 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Initiating leader transfer 01:50:22,435 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-280617298)] resolved 01:50:22,435 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1765595066)] resolved 01:50:22,435 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: successfully resolved all cohort actors 01:50:22,435 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 finishCanCommit 01:50:22,435 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@f25983 01:50:22,435 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-280617298)] 01:50:22,435 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-280617298)] 01:50:22,435 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:50:22,436 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:22,436 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:22,437 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Can committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,437 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,437 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 validated 01:50:22,437 PM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:22,437 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: canCommit for member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: success 01:50:22,437 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown] 01:50:22,438 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:50:22,438 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:22,438 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:22,438 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown] 01:50:22,438 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:50:22,438 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:22,438 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:22,438 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:50:22,438 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1765595066)] 01:50:22,438 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1765595066)] 01:50:22,440 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Can committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,440 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,440 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 validated 01:50:22,440 PM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:22,440 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: canCommit for member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: success 01:50:22,441 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:50:22,441 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit returning result: true 01:50:22,441 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 commit 01:50:22,441 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 finish commit 01:50:22,441 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-280617298)] 01:50:22,441 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-280617298)] 01:50:22,441 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1765595066)] 01:50:22,441 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-1765595066)] 01:50:22,442 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,442 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,442 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 candidate ready 01:50:22,442 PM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:50:22,442 PM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit - messages: [] 01:50:22,442 PM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:50:22,442 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 prepared 01:50:22,442 PM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:22,442 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Finishing commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,442 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 169 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 validated 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: canCommit for member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: success 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 candidate ready 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: canCommit - candidate rootPath: / 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: canCommit - messages: [] 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 prepared 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Finishing commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 scheduled for commit step 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: false 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 submitted to persistence 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:50:22,443 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 228 01:50:22,443 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:22,443 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 candidate ready 01:50:22,443 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 01:50:22,443 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit - messages: [] 01:50:22,443 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:50:22,443 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 prepared 01:50:22,443 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:22,443 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Finishing commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,443 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,443 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,444 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 01:50:22,444 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 submitted to persistence 01:50:22,444 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 173 01:50:22,444 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:50:22,444 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:50:22,444 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:22,444 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:22,444 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 01:50:22,444 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 01:50:22,444 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 01:50:22,444 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 01:50:22,444 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:22,444 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 01:50:22,444 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 01:50:22,444 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 submitted to persistence 01:50:22,445 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:50:22,447 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 01:50:22,447 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 2 01:50:22,447 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:22,447 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=169} 01:50:22,447 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 01:50:22,447 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 01:50:22,447 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:50:22,447 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:22,447 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=228} 01:50:22,447 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=173} 01:50:22,447 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 01:50:22,447 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:50:22,447 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 3 01:50:22,447 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:22,447 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 2 01:50:22,447 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-testLeadershipTransferOnShutdown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:22,447 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 01:50:22,447 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:50:22,447 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 01:50:22,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 01:50:22,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 01:50:22,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 1 01:50:22,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:50:22,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=169} 01:50:22,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,450 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 01:50:22,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:50:22,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:50:22,450 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:22,451 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Setting last applied to 1 01:50:22,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testLeadershipTransferOnShutdown_member-1-shard-cars-testLeadershipTransferOnShutdown%23-280617298$f] 01:50:22,451 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:50:22,451 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=173} 01:50:22,451 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,451 PM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:50:22,451 PM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:50:22,451 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testLeadershipTransferOnShutdown_member-1-shard-people-testLeadershipTransferOnShutdown%23-1765595066$g] 01:50:22,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#1988242231] with name member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 01:50:22,451 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-760717121] with name member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 01:50:22,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 2 01:50:22,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 01:50:22,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=228} 01:50:22,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:50:22,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: commit - successfulFromPrevious: [] 01:50:22,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 01:50:22,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testLeadershipTransferOnShutdown_member-1-shard-cars-testLeadershipTransferOnShutdown$c] 01:50:22,451 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 01:50:22,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@f25983 01:50:22,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 01:50:22,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:22,451 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:22,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:22,451 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 01:50:22,451 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-760717121] 01:50:22,451 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#1988242231] 01:50:22,451 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:50:22,451 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:50:22,451 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-760717121] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:22,451 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#1988242231] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:22,452 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 70.11 μs 01:50:22,452 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 109.1 μs 01:50:22,452 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:50:22,452 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 32.97 μs 01:50:22,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 01:50:22,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:22,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 01:50:22,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:50:22,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-testLeadershipTransferOnShutdown, matchIndex: 2, lastIndex: 2, isVoting: true 01:50:22,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 01:50:22,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:50:22,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:22,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:50:22,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 01:50:22,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 01:50:22,453 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 01:50:22,453 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 2 01:50:22,453 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:22,453 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=169} 01:50:22,453 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 01:50:22,453 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=228} 01:50:22,453 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 01:50:22,453 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 3 01:50:22,453 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:22,453 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 01:50:22,453 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:50:22,453 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:22,453 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=173} 01:50:22,453 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:50:22,453 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 2 01:50:22,453 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testLeadershipTransferOnShutdown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:22,454 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: commit succeeded 01:50:22,454 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 modifications completed with CommitTransactionReply [version=13] 01:50:22,454 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 canCommit 01:50:22,454 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 successfully completed direct commit 01:50:22,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 2 01:50:22,454 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 1 01:50:22,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 01:50:22,454 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:22,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 01:50:22,454 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 01:50:22,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:50:22,454 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:50:22,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=169} 01:50:22,454 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=173} 01:50:22,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,455 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-3-shard-people-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 2 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] is present: existingEntryTerm: 4 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] is present: existingEntryTerm: 4 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 2, entries to be added from: 2 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 3 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 2 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=169} 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1507112156] with name member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 01:50:22,455 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1667624475] with name member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 2 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=228} 01:50:22,455 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#1945524020] with name member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 01:50:22,455 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 2 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=228} 01:50:22,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:50:22,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 01:50:22,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 01:50:22,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 01:50:22,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,456 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 5 01:50:22,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 01:50:22,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] to peer member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:22,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:50:22,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] to peer member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:22,456 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1507112156] 01:50:22,456 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:50:22,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 01:50:22,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:22,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:50:22,456 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:50:22,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 01:50:22,456 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 57.76 μs 01:50:22,456 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:50:22,456 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 31.40 μs 01:50:22,456 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1667624475] 01:50:22,456 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:50:22,456 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 01:50:22,456 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#1945524020] 01:50:22,456 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 01:50:22,456 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:50:22,456 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:50:22,457 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 39.20 μs 01:50:22,457 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:50:22,457 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 35.44 μs 01:50:22,457 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#1945524020] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:22,457 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 119.3 μs 01:50:22,457 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 01:50:22,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:22,459 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:22,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 01:50:22,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:22,459 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 01:50:22,459 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:22,459 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 5 in "RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 4 - updating term 01:50:22,459 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 01:50:22,459 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Term 5 in "RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4]" message is greater than leader's term 4 - switching to Follower 01:50:22,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 01:50:22,459 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 2, lastTerm: 4 01:50:22,459 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 01:50:22,459 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:50:22,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 2, lastTerm: 4 01:50:22,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:50:22,459 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 5 01:50:22,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-475: Closing 01:50:22,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:22,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:50:22,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: removing frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 01:50:22,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown: Closing 01:50:22,459 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 01:50:22,459 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Leader to Follower 01:50:22,459 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Leader to Follower 01:50:22,460 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:50:22,460 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:50:22,460 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 01:50:22,460 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:50:22,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 01:50:22,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:22,460 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:50:22,460 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:50:22,460 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 01:50:22,460 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 01:50:22,460 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 01:50:22,460 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:50:22,460 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:22,460 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:22,460 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 01:50:22,460 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:50:22,461 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown: Closing 01:50:22,461 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-3-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,461 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:50:22,461 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:22,461 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:22,461 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:50:22,461 PM [cluster-test-shard-dispatcher-17] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-3-shard-cars-testLeadershipTransferOnShutdown in 25.80 ms 01:50:22,461 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 01:50:22,461 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-1-shard-cars-testLeadershipTransferOnShutdown status sync done false 01:50:22,461 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Stopping Shard member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:22,461 PM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 01:50:22,461 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:50:22,462 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 1 01:50:22,462 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 01:50:22,462 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:22,462 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 01:50:22,462 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:50:22,462 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=173} 01:50:22,462 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 01:50:22,462 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1204393993] 01:50:22,462 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-3-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,462 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-people-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,462 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 Queuing TransactionOperation 01:50:22,462 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:22,462 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Scheduling 2000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:50:22,462 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done false 01:50:22,462 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1723338405] with name member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 01:50:22,462 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1723338405] 01:50:22,462 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 01:50:22,462 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 01:50:22,462 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:50:22,462 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:50:22,462 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1723338405] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:22,462 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is ready - notifying 1 OnShardInitialized callbacks 01:50:22,463 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 99.45 μs 01:50:22,463 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 01:50:22,463 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:22,463 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:22,463 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:50:22,463 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 01:50:22,463 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2557/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown)] for shard cars 01:50:22,463 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2557/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown)] found - trying create transaction 01:50:22,463 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2557/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown)] 01:50:22,464 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 01:50:22,464 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:50:22,464 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 01:50:22,464 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 1 for term 5 01:50:22,464 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:22,465 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1797172923] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:22,465 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done true 01:50:22,465 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Creating transaction : member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 01:50:22,465 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-2:datastore-testLeadershipTransferOnShutdown@0:2_14#148901206] 01:50:22,467 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-2:datastore-testLeadershipTransferOnShutdown@0:2_14#148901206, transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0, version=13] 01:50:22,467 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:50:22,467 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2557/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-2:datastore-testLeadershipTransferOnShutdown@0:2_14#148901206)] 01:50:22,476 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 ReadData operation succeeded 01:50:22,476 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:50:22,476 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#2036024289] 01:50:22,476 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 01:50:22,476 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:22,476 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1797172923, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@511f8aeb] 01:50:22,477 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1797172923, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@511f8aeb] 01:50:22,477 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractTransactionContextFactory - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1797172923)] for shard cars 01:50:22,477 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 01:50:22,477 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractTransactionContextFactory - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@30b88f67 01:50:22,477 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:22,477 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,477 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-3-shard-people-testLeadershipTransferOnShutdown 01:50:22,477 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 01:50:22,477 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onShutDown 01:50:22,477 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Initiating leader transfer 01:50:22,477 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onShutDown 01:50:22,477 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@65e67f97 01:50:22,477 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@65e67f97 01:50:22,477 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 01:50:22,477 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:22,477 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:50:22,477 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:22,477 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:22,478 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:22,478 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-3-shard-people-testLeadershipTransferOnShutdown 01:50:22,478 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 01:50:22,478 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:50:22,479 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 1 for term 5 01:50:22,479 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown] 01:50:22,479 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:50:22,479 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:22,479 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:22,480 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testLeadershipTransferOnShutdown, matchIndex: 2, lastIndex: 2, isVoting: true 01:50:22,480 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 01:50:22,480 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:22,480 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:22,480 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 01:50:22,483 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 01:50:22,483 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 01:50:22,483 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=6, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:22,483 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 6 01:50:22,483 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,483 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] to peer member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:22,483 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 6 01:50:22,483 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 01:50:22,483 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:22,483 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:50:22,483 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 01:50:22,483 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:50:22,484 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 01:50:22,484 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 01:50:22,485 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:22,485 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 01:50:22,485 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=6, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:22,485 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Term 6 in "RequestVote [term=6, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4]" message is greater than leader's term 5 - switching to Follower 01:50:22,485 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=6, votedFor=null 01:50:22,485 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 01:50:22,485 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=6, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 2, lastTerm: 4 01:50:22,485 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=6, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:22,485 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 01:50:22,485 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 6 01:50:22,485 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-480: Closing 01:50:22,485 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:22,485 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:50:22,485 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: removing frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 01:50:22,485 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 01:50:22,486 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from Leader to Follower 01:50:22,486 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 01:50:22,486 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from Leader to Follower 01:50:22,490 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=true], my term is 6 01:50:22,490 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=true], current voteCount: 1 01:50:22,490 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 01:50:22,490 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:22,490 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:22,490 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 6 01:50:22,490 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 01:50:22,490 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 01:50:22,492 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 01:50:22,492 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:22,492 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-2-shard-cars-testLeadershipTransferOnShutdown in 14.48 ms 01:50:22,492 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 01:50:22,492 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done false 01:50:22,492 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-3-shard-cars-testLeadershipTransferOnShutdown 01:50:22,492 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 01:50:22,492 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:50:22,493 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 01:50:22,493 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 01:50:22,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 01:50:22,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:22,494 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 01:50:22,494 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:50:22,494 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:50:22,494 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:22,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:22,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:22,494 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 01:50:22,494 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 01:50:22,494 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:50:22,494 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-3-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:50:22,494 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-1-shard-cars-testLeadershipTransferOnShutdown 01:50:22,494 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-1-shard-people-testLeadershipTransferOnShutdown 01:50:22,495 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 01:50:22,495 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onShutDown 01:50:22,495 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Initiating leader transfer 01:50:22,495 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:50:22,495 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@58145e97 01:50:22,495 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:22,495 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@58145e97 01:50:22,495 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:22,495 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 01:50:22,495 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:22,495 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:22,496 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown] 01:50:22,501 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:50:22,501 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:22,501 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:22,501 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testLeadershipTransferOnShutdown, matchIndex: 1, lastIndex: 1, isVoting: true 01:50:22,501 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 01:50:22,501 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:22,501 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:22,501 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 01:50:22,502 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 01:50:22,502 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 01:50:22,502 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 01:50:22,502 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 5 01:50:22,502 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-people-testLeadershipTransferOnShutdown 01:50:22,502 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] to peer member-3-shard-people-testLeadershipTransferOnShutdown 01:50:22,502 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:50:22,502 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 01:50:22,502 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:22,503 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:50:22,503 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown: Closing 01:50:22,503 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:50:22,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:22,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 01:50:22,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:22,503 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4]" message is greater than leader's term 4 - switching to Follower 01:50:22,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 01:50:22,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 01:50:22,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 01:50:22,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 01:50:22,503 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 01:50:22,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:50:22,503 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 01:50:22,503 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 5 01:50:22,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageSlicer - member-1-shard-people-testLeadershipTransferOnShutdown (Leader)_slicer-id-476: Closing 01:50:22,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:50:22,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:50:22,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: removing frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 01:50:22,503 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown: Closing 01:50:22,504 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Leader to Follower 01:50:22,504 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 01:50:22,504 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Leader to Follower 01:50:22,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:50:22,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:50:22,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 01:50:22,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:22,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:22,505 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:50:22,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 01:50:22,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: became leader with frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 01:50:22,505 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:50:22,505 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:22,505 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:22,505 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 01:50:22,505 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Candidate to Leader 01:50:22,505 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:50:22,506 PM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown: Closing 01:50:22,506 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:50:22,506 PM [cluster-test-shard-dispatcher-18] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-2-shard-people-testLeadershipTransferOnShutdown in 11.51 ms 01:50:22,506 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 01:50:22,506 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-1-shard-people-testLeadershipTransferOnShutdown status sync done false 01:50:22,507 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Stopping Shard member-1-shard-people-testLeadershipTransferOnShutdown 01:50:22,507 PM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 01:50:22,507 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:50:22,507 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 01:50:22,507 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 01:50:22,508 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:50:22,508 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:50:22,508 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 01:50:22,508 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:50:22,508 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:50:22,509 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 01:50:22,509 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:22,509 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:22,509 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:22,514 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:22,514 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:22,514 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:22,514 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:22,516 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:22,540 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:50:22,541 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testLeadershipTransferOnShutdown 01:50:22,541 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-481: Closing 01:50:22,541 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:50:22,541 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-people-testLeadershipTransferOnShutdown 01:50:22,541 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-2-shard-people-testLeadershipTransferOnShutdown (Leader)_slicer-id-482: Closing 01:50:22,541 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:50:22,541 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 01:50:22,542 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:22,542 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:23,530 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4675357995021967139] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:23,554 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:23,554 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:23,557 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:23.559] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:23,561 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-5586799686954147825] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:23,575 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:23,575 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:23,577 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:23.578] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:23,599 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:23,605 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [1006403785873129215] 01:50:23,605 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:23,606 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:23,606 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:23,607 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:23,607 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:23,619 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:23,619 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:23,620 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:23,626 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:23,632 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [7114834525764173596] 01:50:23,632 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:23,638 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:23,638 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:23,638 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:23,638 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:23,654 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:23,674 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [1887687423493853697] 01:50:23,674 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:23,674 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:23,674 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:23,683 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:23,685 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:23,686 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionForwardedToLeaderAfterRetry 01:50:23,690 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:23,690 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:23,697 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 01:50:23,701 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionForwardedToLeaderAfterRetry 01:50:23,701 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:23,702 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:23,702 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:23,702 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:23,702 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:23,706 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-353962082] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,713 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: no initial generation override, starting from 0 01:50:23,713 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:50:23,713 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2d8568fa to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@22612b6f 01:50:23,713 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@265ebbfb 01:50:23,713 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, 0, 1710165023713, None)) 01:50:23,713 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165023712,0,0)) 01:50:23,713 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@22612b6f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@52f5e12c 01:50:23,714 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} started 01:50:23,714 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2597bc8b 01:50:23,714 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:23,714 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:23,714 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:23,716 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:23,718 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionForwardedToLeaderAfterRetry 01:50:23,718 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:23,718 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:23,718 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486] 01:50:23,718 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:23,718 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:23,718 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:23,718 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-335657112] 01:50:23,718 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, persistent : true 01:50:23,719 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 01:50:23,719 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-335657112] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,719 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#1975706769] 01:50:23,719 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#935302123 created and ready for shard:member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:23,719 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#935302123] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,719 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#1975706769] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,720 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:23,720 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:23,720 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:50:23,720 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:23,720 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [people] 01:50:23,720 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is initialized 01:50:23,720 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 01:50:23,720 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 01:50:23,720 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from null to Follower 01:50:23,722 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:23,724 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, persistent : true 01:50:23,724 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,724 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-2054265201 created and ready for shard:member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:23,724 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-2054265201] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,724 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#-169084248] 01:50:23,725 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#-169084248] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,730 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:23,737 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 01:50:23,737 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:23,740 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:23,740 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:23,740 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:50:23,740 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [cars] 01:50:23,740 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:23,740 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is initialized 01:50:23,740 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 01:50:23,740 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 01:50:23,740 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from null to Follower 01:50:23,741 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 01:50:23,741 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:23,744 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-699323018] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: no initial generation override, starting from 0 01:50:23,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:50:23,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6278daa8 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1e7e223 01:50:23,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@44e57c7b 01:50:23,748 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionForwardedToLeaderAfterRetry 01:50:23,748 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:23,748 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:23,748 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:23,748 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:23,748 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:23,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, 0, 1710165023748, None)) 01:50:23,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165023747,0,0)) 01:50:23,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1e7e223 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4b23388f 01:50:23,749 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2e90e20f 01:50:23,749 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} started 01:50:23,749 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:23,749 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:23,749 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:23,749 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:23,749 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#1868598948] 01:50:23,749 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:23,750 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:23,750 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:23,750 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1765853140] 01:50:23,750 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, persistent : true 01:50:23,750 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, persistent : true 01:50:23,750 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 01:50:23,750 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 01:50:23,750 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#1868598948] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,752 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#1979787097] 01:50:23,754 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#1424233750] 01:50:23,754 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1765853140] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,755 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:23,755 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:23,755 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:23,755 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:23,756 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:50:23,756 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:23,756 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [cars] 01:50:23,756 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:50:23,756 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is initialized 01:50:23,756 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:23,756 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [people] 01:50:23,756 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is initialized 01:50:23,756 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:23,756 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:23,758 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#1507563122 created and ready for shard:member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:23,758 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 01:50:23,758 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 01:50:23,758 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from null to Follower 01:50:23,759 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#1979787097] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,759 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#1424233750] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,759 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#1507563122] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,759 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-194957354 created and ready for shard:member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:23,759 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 01:50:23,759 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 01:50:23,759 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from null to Follower 01:50:23,759 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-194957354] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:23,799 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-353962082] 01:50:23,799 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:23,799 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486] 01:50:23,995 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:23,995 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Received ElectionTimeout - switching to Candidate 01:50:23,995 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 01:50:23,995 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:23,995 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 1 01:50:23,995 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:23,995 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:50:23,995 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:50:23,995 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:23,996 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 01:50:23,996 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 01:50:24,15 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:24,15 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Received ElectionTimeout - switching to Candidate 01:50:24,15 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 01:50:24,15 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=1, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:24,15 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 1 01:50:24,15 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:24,15 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:50:24,15 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:50:24,15 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:24,15 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 01:50:24,16 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 01:50:24,297 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:24,305 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:50:24,305 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:24,305 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 2 01:50:24,307 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:24,335 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:50:24,335 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=2, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:24,335 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 2 01:50:24,586 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:50:24,586 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:24,586 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 3 01:50:24,625 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:50:24,625 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:24,625 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 3 01:50:24,626 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:24,626 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:24,626 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:24,627 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:24,627 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:24,627 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:24,627 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:24,627 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:24,627 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:24,627 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:24,627 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:24,627 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:24,627 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:24,627 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:24,634 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:24,635 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:24,635 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:24,635 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:24,635 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:24,635 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:24,635 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:24,635 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:24,635 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:24,635 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:24,698 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:24,815 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:50:24,815 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:24,815 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 4 01:50:24,815 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:24,817 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:24,817 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 01:50:24,817 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:24,817 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:24,817 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=null 01:50:24,817 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:24,817 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:24,817 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:24,819 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:24,819 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:24,819 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 01:50:24,819 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:24,819 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:24,819 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 01:50:24,819 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [] 01:50:24,819 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 01:50:24,819 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:50:24,819 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:24,819 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:24,819 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 01:50:24,820 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:24,820 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-1-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:24,820 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:50:24,820 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:24,820 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:24,821 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done false 01:50:24,821 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: -1, nextIndex: 0 01:50:24,821 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:24,937 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done true 01:50:24,945 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:50:24,945 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:24,945 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 4 01:50:24,945 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:24,947 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:24,947 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 01:50:24,947 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:24,947 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:24,947 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=null 01:50:24,947 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:24,947 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:24,947 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:24,948 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:24,948 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:24,949 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 01:50:24,949 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:24,949 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:24,949 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 01:50:24,949 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [] 01:50:24,949 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 01:50:24,949 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:50:24,949 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:24,949 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:24,949 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 01:50:24,949 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 01:50:24,950 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:24,950 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:24,950 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:50:24,950 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:24,950 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:24,950 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 01:50:24,950 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done false 01:50:24,951 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: -1, nextIndex: 0 01:50:24,951 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:25,9 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-353962082] 01:50:25,9 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 01:50:25,9 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-335657112] 01:50:25,11 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-353962082] 01:50:25,11 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:25,11 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486] 01:50:25,11 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486] 01:50:25,11 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-699323018] 01:50:25,12 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:25,12 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#1868598948] 01:50:25,12 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#1868598948] 01:50:25,12 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionForwardedToLeaderAfterRetry commit rateLimit adjusted to 162.12298098119095 01:50:25,12 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,13 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-699323018] 01:50:25,13 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 01:50:25,13 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:25,13 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-699323018] 01:50:25,13 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:50:25,13 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 01:50:25,13 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Readying 2 components for commit 01:50:25,13 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:50:25,13 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Readying transaction for shard cars 01:50:25,13 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:50:25,13 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 01:50:25,13 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:50:25,13 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Readying transaction for shard people 01:50:25,13 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 01:50:25,13 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 canCommit 01:50:25,13 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:50:25,13 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars 01:50:25,13 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 01:50:25,13 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:25,13 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 = [ 1 ] 01:50:25,13 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 readyTransaction called 01:50:25,13 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 sending 1 batched modifications, ready: true 01:50:25,13 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] for shard people 01:50:25,13 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,13 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 01:50:25,13 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 01:50:25,13 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 = [ 1 ] 01:50:25,13 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 readyTransaction called 01:50:25,13 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 sending 1 batched modifications, ready: true 01:50:25,13 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,16 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, disabling transaction tracking 01:50:25,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, disabling transaction tracking 01:50:25,16 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:50:25,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:50:25,16 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:50:25,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:50:25,16 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:25,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:25,16 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:25,16 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:25,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:25,16 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 01:50:25,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 01:50:25,16 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,16 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 of 1 operations, client version 13 01:50:25,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 of 1 operations, client version 13 01:50:25,16 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,16 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 ready 01:50:25,16 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 ready 01:50:25,17 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:50:25,17 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:50:25,18 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 readyTransaction 01:50:25,18 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:50:25,18 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 readyTransaction 01:50:25,19 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486)] resolved 01:50:25,19 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-335657112)] resolved 01:50:25,19 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:50:25,19 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: successfully resolved all cohort actors 01:50:25,19 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 01:50:25,19 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:50:25,19 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 finishCanCommit 01:50:25,19 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:25,19 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 01:50:25,19 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:50:25,19 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486)] 01:50:25,19 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486)] 01:50:25,19 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 1 01:50:25,19 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:25,19 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 01:50:25,19 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:50:25,19 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:25,19 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 01:50:25,19 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:50:25,19 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 1 01:50:25,19 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:25,20 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:50:25,20 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:50:25,20 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done true 01:50:25,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 0, nextIndex: 1 01:50:25,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:25,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 0 01:50:25,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 01:50:25,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 01:50:25,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:50:25,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 01:50:25,20 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 0, nextIndex: 1 01:50:25,20 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=4783942} is already disabled 01:50:25,21 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:25,21 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 0 01:50:25,21 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 01:50:25,21 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:25,21 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 01:50:25,21 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:50:25,21 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 01:50:25,21 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=4928066} is already disabled 01:50:25,21 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 01:50:25,21 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,21 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:25,21 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,21 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 validated 01:50:25,21 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:25,21 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: success 01:50:25,21 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 01:50:25,22 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 0 01:50:25,22 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:25,22 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 0 01:50:25,22 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 0 01:50:25,22 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:25,22 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 01:50:25,22 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 0 01:50:25,22 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 01:50:25,22 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 01:50:25,22 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 01:50:25,22 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:50:25,22 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:50:25,22 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 01:50:25,22 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 01:50:25,22 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 01:50:25,22 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 01:50:25,23 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:50:25,23 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-335657112)] 01:50:25,23 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-335657112)] 01:50:25,28 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,28 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,28 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 validated 01:50:25,28 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:25,28 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: success 01:50:25,29 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:50:25,29 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit returning result: true 01:50:25,29 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 commit 01:50:25,29 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 finish commit 01:50:25,29 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486)] 01:50:25,29 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486)] 01:50:25,29 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-335657112)] 01:50:25,29 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-335657112)] 01:50:25,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 candidate ready 01:50:25,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:50:25,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit - messages: [] 01:50:25,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:50:25,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 prepared 01:50:25,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:25,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 179 01:50:25,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:50:25,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:50:25,30 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:25,30 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:25,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,30 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 candidate ready 01:50:25,30 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:50:25,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 01:50:25,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 submitted to persistence 01:50:25,30 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit - messages: [] 01:50:25,30 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:50:25,30 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 prepared 01:50:25,30 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:25,30 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,30 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,30 PM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 183 01:50:25,31 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 01:50:25,31 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:50:25,31 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 01:50:25,31 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:25,31 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:25,31 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:25,31 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}]]] 01:50:25,31 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 submitted to persistence 01:50:25,31 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 01:50:25,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 01:50:25,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:50:25,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:25,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=179} 01:50:25,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:50:25,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 01:50:25,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}]]] 01:50:25,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:25,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:50:25,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:25,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=183} 01:50:25,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 01:50:25,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 01:50:25,32 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:25,33 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 01:50:25,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:50:25,33 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 01:50:25,33 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 01:50:25,33 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:25,33 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:25,33 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 1 01:50:25,33 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 1 01:50:25,33 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 01:50:25,33 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 01:50:25,33 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data Simple{size=179} 01:50:25,33 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data Simple{size=183} 01:50:25,33 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,33 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,33 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:50:25,33 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:50:25,33 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:50:25,33 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:50:25,33 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-people-testTransactionForwardedToLeaderAfterRetry%23-335657112$f] 01:50:25,33 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry%23147759486$e] 01:50:25,34 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1191293782] with name member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 01:50:25,34 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#916974035] with name member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 01:50:25,34 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1191293782] 01:50:25,34 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:25,34 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 01:50:25,34 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 01:50:25,34 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:25,34 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:25,34 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 01:50:25,34 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 101.4 μs 01:50:25,34 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 01:50:25,34 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#916974035] 01:50:25,34 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1191293782] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:25,34 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:25,34 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#916974035] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:25,34 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 77.95 μs 01:50:25,37 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: commit succeeded 01:50:25,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 1 01:50:25,37 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 1 01:50:25,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:25,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 1 01:50:25,37 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:25,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 01:50:25,37 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 1 01:50:25,37 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 01:50:25,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data Simple{size=183} 01:50:25,37 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data Simple{size=179} 01:50:25,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,37 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,37 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1028237327] with name member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 01:50:25,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#548359888] with name member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 01:50:25,37 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 01:50:25,37 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 01:50:25,37 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 01:50:25,38 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 01:50:25,38 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1028237327] 01:50:25,38 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#548359888] 01:50:25,38 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:25,38 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:25,38 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 89.84 μs 01:50:25,38 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1028237327] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:25,38 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#548359888] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:25,38 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 143.0 μs 01:50:25,39 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-699323018] 01:50:25,39 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:25,39 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#1868598948] 01:50:25,40 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#1868598948] 01:50:25,41 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-699323018] 01:50:25,41 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 01:50:25,41 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1765853140] 01:50:25,41 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1765853140] 01:50:25,41 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,41 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:25,41 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:25,41 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 01:50:25,41 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:50:25,41 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] for shard people, trying to use DirectTransactionContextWrapper 01:50:25,41 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:25,41 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 01:50:25,41 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 01:50:25,41 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Readying 2 components for commit 01:50:25,41 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Readying transaction for shard cars 01:50:25,41 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 = [ 1 ] 01:50:25,41 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 readyTransaction called 01:50:25,41 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 sending 1 batched modifications, ready: true 01:50:25,41 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,42 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Readying transaction for shard people 01:50:25,42 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 = [ 1 ] 01:50:25,42 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 readyTransaction called 01:50:25,42 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 sending 1 batched modifications, ready: true 01:50:25,42 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,42 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 canCommit 01:50:25,43 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,43 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,43 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 of 1 operations, client version 13 01:50:25,43 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 of 1 operations, client version 13 01:50:25,43 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,43 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,43 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 ready 01:50:25,43 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 ready 01:50:25,44 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 readyTransaction 01:50:25,44 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-335657112)] resolved 01:50:25,44 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 modifications completed with ReadyTransactionReply [version=13] 01:50:25,44 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 readyTransaction 01:50:25,44 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486)] resolved 01:50:25,44 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: successfully resolved all cohort actors 01:50:25,44 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 finishCanCommit 01:50:25,44 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486)] 01:50:25,44 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486)] 01:50:25,44 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 modifications completed with ReadyTransactionReply [version=13] 01:50:25,44 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,44 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,44 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 validated 01:50:25,45 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:25,45 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: success 01:50:25,48 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:50:25,48 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-335657112)] 01:50:25,48 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-335657112)] 01:50:25,49 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,49 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,49 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 validated 01:50:25,49 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:25,49 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: success 01:50:25,49 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:50:25,49 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: canCommit returning result: true 01:50:25,49 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,50 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:25,50 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:25,50 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 01:50:25,50 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}] 01:50:25,50 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] for shard people, trying to use DirectTransactionContextWrapper 01:50:25,50 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:25,50 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 01:50:25,50 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person 01:50:25,50 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Readying 2 components for commit 01:50:25,50 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Readying transaction for shard cars 01:50:25,50 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 = [ 1 ] 01:50:25,50 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 readyTransaction called 01:50:25,50 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 sending 1 batched modifications, ready: true 01:50:25,50 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,50 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Readying transaction for shard people 01:50:25,50 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 = [ 1 ] 01:50:25,50 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 readyTransaction called 01:50:25,50 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 sending 1 batched modifications, ready: true 01:50:25,50 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,50 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-353962082] 01:50:25,50 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:25,50 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486] 01:50:25,50 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486] 01:50:25,50 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-699323018] 01:50:25,51 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:25,51 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#1868598948] 01:50:25,51 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#1868598948] 01:50:25,52 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:50:25,52 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:25,52 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:25,52 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 01:50:25,52 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}] 01:50:25,52 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}] 01:50:25,52 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 sending 2 batched modifications, ready: false 01:50:25,52 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,52 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}] 01:50:25,52 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car5}] 01:50:25,54 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,54 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 of 1 operations, client version 13 01:50:25,54 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,54 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 ready 01:50:25,55 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,55 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 of 1 operations, client version 13 01:50:25,55 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,55 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 ready 01:50:25,55 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 2 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:50:25,56 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 readyTransaction 01:50:25,56 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 modifications completed with ReadyTransactionReply [version=13] 01:50:25,56 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 readyTransaction 01:50:25,56 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 modifications completed with ReadyTransactionReply [version=13] 01:50:25,56 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 modifications completed with BatchedModificationsReply [numBatched=2] 01:50:25,57 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 sending 2 batched modifications, ready: false 01:50:25,57 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,57 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car6}] 01:50:25,57 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 01:50:25,57 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:25,57 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:25,57 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 01:50:25,57 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car7}] 01:50:25,57 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:50:25,58 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:25,58 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:25,58 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - trying create transaction 01:50:25,58 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,58 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 Queuing TransactionOperation 01:50:25,58 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-353962082] 01:50:25,58 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:25,58 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486] 01:50:25,58 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486] 01:50:25,59 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 2 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:50:25,59 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Creating transaction : member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:50:25,59 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-2:datastore-testTransactionForwardedToLeaderAfterRetry@0:5_15#173283515] 01:50:25,60 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-2:datastore-testTransactionForwardedToLeaderAfterRetry@0:5_15#173283515] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:25,60 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 modifications completed with BatchedModificationsReply [numBatched=2] 01:50:25,60 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-2:datastore-testTransactionForwardedToLeaderAfterRetry@0:5_15#173283515, transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0, version=13] 01:50:25,60 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car8}] 01:50:25,108 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486] 01:50:25,108 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-699323018] 01:50:25,109 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:25,109 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#1868598948] 01:50:25,109 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#1868598948] 01:50:25,109 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionForwardedToLeaderAfterRetry 01:50:25,110 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,110 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:25,110 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config null, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:25,110 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config null, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:25,110 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-490: Closing 01:50:25,110 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:50:25,110 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:25,110 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-489: Closing 01:50:25,110 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:50:25,110 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: removing frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 01:50:25,110 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:25,110 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:25,110 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: removing frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 01:50:25,110 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:25,110 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Leader to Follower 01:50:25,110 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Leader to Follower 01:50:25,110 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 01:50:25,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:25,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:25,111 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Leader to Follower 01:50:25,111 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 01:50:25,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:25,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:25,111 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Leader to Follower 01:50:25,160 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-353962082] 01:50:25,160 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:25,160 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486] 01:50:25,160 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 canCommit 01:50:25,160 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 Readying 1 components for commit 01:50:25,160 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 Readying transaction for shard cars 01:50:25,160 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486)] resolved 01:50:25,160 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 directCommit called 01:50:25,160 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 sending 1 batched modifications, ready: true 01:50:25,160 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-335657112)] resolved 01:50:25,160 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: successfully resolved all cohort actors 01:50:25,160 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,160 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 finishCanCommit 01:50:25,160 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486)] 01:50:25,160 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 Readying 1 components for commit 01:50:25,160 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486)] 01:50:25,160 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 Readying transaction for shard cars 01:50:25,160 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 directCommit called 01:50:25,160 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 sending 1 batched modifications, ready: true 01:50:25,160 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,160 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 Readying 1 components for commit 01:50:25,160 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 Readying transaction for shard cars 01:50:25,160 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 directCommit called 01:50:25,160 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 sending 1 batched modifications, ready: true 01:50:25,161 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-2:datastore-testTransactionForwardedToLeaderAfterRetry@0:5_15#173283515)] 01:50:25,161 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionForwardedToLeaderAfterRetry 01:50:25,161 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,161 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:25,161 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 01:50:25,161 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,161 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 01:50:25,162 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Re-initializing to Follower with previous leaderId member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:25,162 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:50:25,162 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : true 01:50:25,162 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:25,162 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:50:25,162 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : true 01:50:25,162 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:25,165 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Adding message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0, ready=true, participatingShardNames=null, totalMessagesSent=3, modifications size=1] to retry 01:50:25,165 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,166 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Adding message CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, version=13] to retry 01:50:25,166 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Adding message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0, ready=true, participatingShardNames=null, totalMessagesSent=1, modifications size=1] to retry 01:50:25,166 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:50:25,166 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: handleForwardedReadyTransaction for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:50:25,167 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Adding message ForwardedReadyTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0, transaction=ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0, closed=false, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car8}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car8}], operation=WRITE}}}}}}}]}, doImmediateCommit=true, participatingShardNames=null, txnClientVersion=13] to retry 01:50:25,211 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-699323018] 01:50:25,211 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:25,211 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#1868598948] 01:50:25,211 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Received TimeoutNow - switching to Candidate 01:50:25,212 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry] 01:50:25,212 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,212 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 5 01:50:25,212 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,212 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:50:25,212 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:50:25,212 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:50:25,212 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:25,212 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:25,212 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 01:50:25,212 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 01:50:25,212 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:25,212 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:25,212 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 01:50:25,213 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:25,213 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 01:50:25,213 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:25,213 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 4 - updating term 01:50:25,213 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=null 01:50:25,213 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 01:50:25,213 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,213 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:50:25,215 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:50:25,215 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:50:25,215 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry] 01:50:25,215 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:25,215 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:50:25,215 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 01:50:25,215 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 01:50:25,215 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 01:50:25,215 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:50:25,215 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:25,215 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:25,215 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 01:50:25,216 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 01:50:25,216 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:25,216 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding 3 pending transaction messages to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding pending transaction message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, ready=true, participatingShardNames=[cars, people], totalMessagesSent=1, modifications size=1] 01:50:25,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding pending transaction message CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] 01:50:25,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding pending transaction message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, ready=true, participatingShardNames=[cars, people], totalMessagesSent=1, modifications size=1] 01:50:25,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Retrying message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0, ready=true, participatingShardNames=null, totalMessagesSent=3, modifications size=1] 01:50:25,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Retrying message CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, version=13] 01:50:25,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Retrying message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0, ready=true, participatingShardNames=null, totalMessagesSent=1, modifications size=1] 01:50:25,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Retrying message ForwardedReadyTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0, transaction=ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0, closed=false, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car8}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car8}], operation=WRITE}}}}}}}]}, doImmediateCommit=true, participatingShardNames=null, txnClientVersion=13] 01:50:25,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,221 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding 3 BatchedModifications to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,221 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,221 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,221 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding CanCommitTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,221 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,221 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding 1 BatchedModifications to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,221 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: handleForwardedReadyTransaction for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:50:25,221 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,221 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:50:25,221 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,221 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:25,221 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:25,222 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done false 01:50:25,223 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 01:50:25,223 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:25,223 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,223 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 of 1 operations, client version 13 01:50:25,223 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,223 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 ready 01:50:25,223 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,223 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,223 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 validated 01:50:25,223 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:25,223 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: success 01:50:25,223 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,224 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 of 1 operations, client version 13 01:50:25,224 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,224 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 ready 01:50:25,224 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done true 01:50:25,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 2 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:50:25,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 2 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:50:25,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:50:25,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 of 5 operations, client version 13 01:50:25,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:50:25,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 ready 01:50:25,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 is scheduled for canCommit step 01:50:25,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 is scheduled for canCommit step 01:50:25,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 01:50:25,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 of 1 operations, client version 13 01:50:25,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 01:50:25,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 ready 01:50:25,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 is scheduled for canCommit step 01:50:25,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:50:25,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 of 1 operations, client version 13 01:50:25,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:50:25,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 ready 01:50:25,225 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 modifications completed with BatchedModificationsReply [numBatched=2] 01:50:25,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 is scheduled for canCommit step 01:50:25,226 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.datastore.messages.BatchedModificationsReply] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#1868598948] to Actor[akka://cluster-test/deadLetters] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:25,262 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-699323018] 01:50:25,262 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 01:50:25,262 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1765853140] 01:50:25,262 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 commit 01:50:25,262 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Received TimeoutNow - switching to Candidate 01:50:25,262 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 finish commit 01:50:25,262 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testTransactionForwardedToLeaderAfterRetry] 01:50:25,262 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:25,262 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486)] 01:50:25,262 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486)] 01:50:25,262 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 5 01:50:25,262 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:25,262 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-335657112)] 01:50:25,262 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-335657112)] 01:50:25,262 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:50:25,262 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:50:25,262 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:50:25,262 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:50:25,262 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:25,262 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 01:50:25,263 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 01:50:25,263 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:25,263 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:25,263 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 01:50:25,263 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,263 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:25,263 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 01:50:25,263 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:25,263 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 4 - updating term 01:50:25,263 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=null 01:50:25,263 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding CommitTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,263 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 01:50:25,263 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:25,264 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:50:25,264 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Adding message CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] to retry 01:50:25,265 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:50:25,265 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:50:25,265 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-1-shard-people-testTransactionForwardedToLeaderAfterRetry] 01:50:25,265 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,265 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:25,265 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,265 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:50:25,265 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 01:50:25,265 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 01:50:25,265 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 01:50:25,265 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 candidate ready 01:50:25,265 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: canCommit - candidate rootPath: / 01:50:25,265 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:50:25,265 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:25,265 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 01:50:25,265 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:25,265 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 01:50:25,265 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 01:50:25,265 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: canCommit - messages: [] 01:50:25,266 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 01:50:25,266 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 prepared 01:50:25,266 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:25,266 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,266 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,266 PM [cluster-test-shard-dispatcher-26] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 77 01:50:25,266 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,266 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 validated 01:50:25,266 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:25,266 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: success 01:50:25,266 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:50:25,266 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}] 01:50:25,266 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}] 01:50:25,266 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-335657112)] 01:50:25,266 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-335657112)] 01:50:25,266 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:25,266 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:50:25,266 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 2 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,266 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}]]] 01:50:25,266 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 submitted to persistence 01:50:25,267 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}] 01:50:25,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:25,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:25,276 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:50:25,276 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:25,276 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:25,276 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 01:50:25,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding 3 pending transaction messages to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding pending transaction message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, ready=true, participatingShardNames=[cars, people], totalMessagesSent=1, modifications size=1] 01:50:25,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding pending transaction message CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] 01:50:25,276 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}]]] 01:50:25,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding pending transaction message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, ready=true, participatingShardNames=[cars, people], totalMessagesSent=1, modifications size=1] 01:50:25,276 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:50:25,276 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:50:25,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Retrying message CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] 01:50:25,276 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=77} 01:50:25,276 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}] 01:50:25,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:25,276 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 01:50:25,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding CanCommitTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,276 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:25,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:25,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding CommitTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,276 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done false 01:50:25,279 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}] 01:50:25,281 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 01:50:25,281 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 2, nextIndex: 3 01:50:25,281 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:50:25,281 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 2 01:50:25,281 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 01:50:25,281 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:25,281 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=77} 01:50:25,281 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,281 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: commit - successfulFromPrevious: [] 01:50:25,281 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 01:50:25,281 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry%23147759486$t] 01:50:25,282 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,282 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 01:50:25,282 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:25,282 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 01:50:25,282 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:25,282 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 of 1 operations, client version 13 01:50:25,282 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 25.28 μs 01:50:25,282 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,282 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 ready 01:50:25,282 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,282 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,282 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 validated 01:50:25,282 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:25,282 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: success 01:50:25,283 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,283 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 of 1 operations, client version 13 01:50:25,283 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,283 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 ready 01:50:25,283 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,283 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Evaluating tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 for canCommit - preceding participating shard names [cars] 01:50:25,283 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Skipping pending transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 in state CAN_COMMIT_COMPLETE 01:50:25,283 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Not moving tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 - cannot proceed with canCommit 01:50:25,283 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,283 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,283 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 candidate ready 01:50:25,283 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: canCommit - candidate rootPath: / 01:50:25,283 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: canCommit - messages: [] 01:50:25,283 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 01:50:25,283 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 prepared 01:50:25,283 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:25,283 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,283 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,284 PM [cluster-test-shard-dispatcher-26] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 184 01:50:25,284 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,284 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 validated 01:50:25,284 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:25,284 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: success 01:50:25,284 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:50:25,284 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: canCommit returning result: true 01:50:25,284 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}] 01:50:25,284 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}] 01:50:25,284 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:25,284 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:50:25,284 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 2 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:25,284 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}]]] 01:50:25,284 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 submitted to persistence 01:50:25,285 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}] 01:50:25,286 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 2 01:50:25,286 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:50:25,286 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 2 01:50:25,286 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 01:50:25,286 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=77} 01:50:25,286 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}]]] 01:50:25,286 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:50:25,286 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:50:25,286 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=184} 01:50:25,286 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}] 01:50:25,286 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,286 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 01:50:25,286 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:25,286 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 01:50:25,286 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 01:50:25,286 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:25,286 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 22.48 μs 01:50:25,286 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}] 01:50:25,287 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done true 01:50:25,287 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 2 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:25,287 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 2, nextIndex: 3 01:50:25,287 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:50:25,287 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 2 01:50:25,287 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 01:50:25,287 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=184} 01:50:25,287 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,287 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: commit - successfulFromPrevious: [] 01:50:25,287 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 01:50:25,287 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-people-testTransactionForwardedToLeaderAfterRetry%23-335657112$u] 01:50:25,287 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 01:50:25,287 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:25,287 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 01:50:25,288 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: commit succeeded 01:50:25,288 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:25,288 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 commit 01:50:25,288 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 finish commit 01:50:25,288 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 80.24 μs 01:50:25,288 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486)] 01:50:25,288 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486)] 01:50:25,288 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-335657112)] 01:50:25,288 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-335657112)] 01:50:25,291 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 2 01:50:25,291 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:50:25,292 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 2 01:50:25,292 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,292 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 01:50:25,292 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=184} 01:50:25,292 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding CommitTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,292 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:50:25,292 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 01:50:25,292 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 01:50:25,292 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:25,292 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding CommitTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,292 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:25,292 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 101.3 μs 01:50:25,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,293 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,293 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 candidate ready 01:50:25,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: canCommit - candidate rootPath: / 01:50:25,293 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 candidate ready 01:50:25,293 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: canCommit - candidate rootPath: / 01:50:25,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: canCommit - messages: [] 01:50:25,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 01:50:25,293 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: canCommit - messages: [] 01:50:25,293 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 01:50:25,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 prepared 01:50:25,293 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 prepared 01:50:25,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:25,293 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:25,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,293 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,293 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,293 PM [cluster-test-shard-dispatcher-26] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:50:25,293 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}] 01:50:25,293 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}] 01:50:25,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 236 01:50:25,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:50:25,293 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:25,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 validated 01:50:25,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:25,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: success 01:50:25,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:50:25,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:50:25,294 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 01:50:25,294 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:25,294 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}]]] 01:50:25,294 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 submitted to persistence 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 candidate ready 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: canCommit - candidate rootPath: / 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: canCommit - messages: [] 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: preCommit - successfulFromPrevious: [] 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 prepared 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 scheduled for commit step 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 validated 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: success 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 candidate ready 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: canCommit - candidate rootPath: / 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: canCommit - messages: [] 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: preCommit - successfulFromPrevious: [] 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 prepared 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 scheduled for commit step 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 validated 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: success 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 candidate ready 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: canCommit - candidate rootPath: / 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: canCommit - messages: [] 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: preCommit - successfulFromPrevious: [] 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 prepared 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 scheduled for commit step 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}] 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}] 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: false 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 submitted to persistence 01:50:25,294 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:50:25,295 PM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 324 01:50:25,295 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}] 01:50:25,295 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}] 01:50:25,295 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: false 01:50:25,295 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 submitted to persistence 01:50:25,295 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 01:50:25,295 PM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 236 01:50:25,295 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}] 01:50:25,295 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}] 01:50:25,295 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: false 01:50:25,295 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 submitted to persistence 01:50:25,295 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:50:25,295 PM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 236 01:50:25,295 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}] 01:50:25,295 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}] 01:50:25,295 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:25,295 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 6, leaderSnapShotIndex: 1 01:50:25,295 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,295 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}], SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}], SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}], SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}]]] 01:50:25,295 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}] 01:50:25,295 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 submitted to persistence 01:50:25,296 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}] 01:50:25,296 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}] 01:50:25,296 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}] 01:50:25,296 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}] 01:50:25,299 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}]]] 01:50:25,299 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:50:25,299 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}], SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}], SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}], SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}]]] 01:50:25,299 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:50:25,299 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=222} 01:50:25,299 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 4 01:50:25,299 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}] 01:50:25,299 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:50:25,299 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=236} 01:50:25,299 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}] 01:50:25,299 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 01:50:25,299 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=3, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:25,299 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=324} 01:50:25,299 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}] 01:50:25,299 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=236} 01:50:25,299 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}] 01:50:25,299 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=236} 01:50:25,299 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}] 01:50:25,299 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 5 01:50:25,299 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=6, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:25,299 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}] 01:50:25,299 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}] 01:50:25,299 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}] 01:50:25,299 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}] 01:50:25,299 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}] 01:50:25,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 3, nextIndex: 4 01:50:25,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 01:50:25,300 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 6, nextIndex: 7 01:50:25,300 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 2 01:50:25,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 3 01:50:25,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 01:50:25,300 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 3 01:50:25,300 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 01:50:25,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data Simple{size=222} 01:50:25,300 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data Simple{size=236} 01:50:25,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,300 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: commit - successfulFromPrevious: [] 01:50:25,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 01:50:25,300 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: commit - successfulFromPrevious: [] 01:50:25,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-people-testTransactionForwardedToLeaderAfterRetry%23-335657112$x] 01:50:25,300 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 01:50:25,300 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry%23147759486$w] 01:50:25,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 2 for term 5 01:50:25,301 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:50:25,301 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=3 01:50:25,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 4 01:50:25,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 01:50:25,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=324} 01:50:25,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:50:25,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: commit - successfulFromPrevious: [] 01:50:25,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 changing state from COMMIT_PENDING to COMMITTED 01:50:25,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry$p] 01:50:25,301 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: commit succeeded 01:50:25,301 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:25,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 5 01:50:25,301 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 65.67 μs 01:50:25,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 01:50:25,301 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:25,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data Simple{size=236} 01:50:25,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 01:50:25,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: commit - successfulFromPrevious: [] 01:50:25,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 changing state from COMMIT_PENDING to COMMITTED 01:50:25,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry$r] 01:50:25,301 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 32.29 μs 01:50:25,301 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 modifications completed with CommitTransactionReply [version=13] 01:50:25,301 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:25,301 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 32.37 μs 01:50:25,301 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:25,301 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 29.94 μs 01:50:25,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 6 01:50:25,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 5 to 6 01:50:25,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 6 data Simple{size=236} 01:50:25,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:50:25,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: commit - successfulFromPrevious: [] 01:50:25,302 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 changing state from COMMIT_PENDING to COMMITTED 01:50:25,302 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry_shard-cars-member-2%3Adatastore-testTransactionForwardedToLeaderAfterRetry%400%3A5_15%23173283515$s] 01:50:25,302 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 modifications completed with CommitTransactionReply [version=13] 01:50:25,302 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:25,302 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 33.88 μs 01:50:25,302 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 5 for term 5 01:50:25,302 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 01:50:25,302 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=6 01:50:25,302 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 canCommit 01:50:25,302 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 successfully completed direct commit 01:50:25,303 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 canCommit 01:50:25,303 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 successfully completed direct commit 01:50:25,303 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 canCommit 01:50:25,303 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 successfully completed direct commit 01:50:25,303 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-353962082] 01:50:25,303 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:25,303 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 3 01:50:25,303 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 6 01:50:25,303 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 01:50:25,303 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 3 01:50:25,303 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 01:50:25,303 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 2, lastIndex(): 6 01:50:25,303 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data Simple{size=222} 01:50:25,303 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 3 01:50:25,303 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 01:50:25,303 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data Simple{size=236} 01:50:25,303 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486] 01:50:25,303 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,304 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:50:25,304 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 2 for term 5 01:50:25,304 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 4 01:50:25,304 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 01:50:25,304 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=324} 01:50:25,304 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=3 01:50:25,304 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#147759486] 01:50:25,304 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:50:25,305 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 5 01:50:25,305 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 01:50:25,305 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data Simple{size=236} 01:50:25,305 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 01:50:25,305 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 6 01:50:25,305 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 5 to 6 01:50:25,305 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 6 data Simple{size=236} 01:50:25,305 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:50:25,305 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 5 for term 5 01:50:25,305 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=6 01:50:25,305 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:25,306 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 120.3 μs 01:50:25,306 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:25,306 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 41.87 μs 01:50:25,306 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:25,306 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 42.64 μs 01:50:25,306 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:25,306 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 37.46 μs 01:50:25,306 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:50:25,306 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 42.60 μs 01:50:25,307 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-699323018] 01:50:25,307 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:50:25,307 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#1868598948] 01:50:25,307 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#1868598948] 01:50:25,307 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,308 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-353962082] 01:50:25,308 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 01:50:25,308 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:25,308 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:50:25,308 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:50:25,308 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars 01:50:25,308 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - trying create transaction 01:50:25,308 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,309 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, disabling transaction tracking 01:50:25,309 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:50:25,309 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:50:25,309 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:25,309 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: 5 01:50:25,309 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 7 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,309 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=6, prevLogTerm=5, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 01:50:25,309 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Creating transaction : member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,309 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_16#-1162200257] 01:50:25,310 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:50:25,311 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=6, prevLogTerm=5, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 01:50:25,311 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:50:25,311 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 01:50:25,311 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 01:50:25,311 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:50:25,311 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 01:50:25,311 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=7, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:25,311 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] RemoteTransactionContextSupport - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_16#-1162200257, transactionId=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] 01:50:25,311 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:50:25,311 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] RemoteTransactionContext - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:50:25,312 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_16#-1162200257)] 01:50:25,312 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 7, nextIndex: 8 01:50:25,312 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 6 01:50:25,312 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 7 01:50:25,313 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 6 to 7 01:50:25,313 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 7 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 01:50:25,313 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:50:25,313 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 01:50:25,313 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=3749746} is already disabled 01:50:25,313 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 6 for term 5 01:50:25,313 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 01:50:25,313 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=7 01:50:25,314 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 7 01:50:25,314 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 7, context.getLastApplied(): 6, lastIndex(): 7 01:50:25,314 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 7 01:50:25,314 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 6 to 7 01:50:25,314 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 7 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 01:50:25,314 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:50:25,314 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 01:50:25,314 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 6 for term 5 01:50:25,315 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=7 01:50:25,316 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] RemoteTransactionContext - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 ReadData operation succeeded 01:50:25,316 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-353962082] 01:50:25,316 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 01:50:25,317 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:50:25,317 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:50:25,317 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:50:25,317 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] for shard people 01:50:25,317 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] RemoteTransactionContextSupport - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] found - trying create transaction 01:50:25,317 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] 01:50:25,318 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, disabling transaction tracking 01:50:25,318 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:50:25,318 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:50:25,318 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:25,318 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 2 01:50:25,318 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 4 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:25,319 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=3, prevLogTerm=5, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 01:50:25,319 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Creating transaction : member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:50:25,319 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-people-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_17#574230587] 01:50:25,319 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:50:25,320 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=3, prevLogTerm=5, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 01:50:25,320 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:50:25,320 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 01:50:25,320 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] RemoteTransactionContextSupport - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-people-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_17#574230587, transactionId=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] 01:50:25,320 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 01:50:25,320 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:50:25,320 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 01:50:25,320 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] RemoteTransactionContext - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person 01:50:25,320 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=4, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:25,320 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-people-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_17#574230587)] 01:50:25,320 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 01:50:25,321 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 4, nextIndex: 5 01:50:25,321 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 01:50:25,321 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 4 01:50:25,321 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 01:50:25,321 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 01:50:25,321 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:50:25,321 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 01:50:25,321 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=2708629} is already disabled 01:50:25,321 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 01:50:25,321 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:50:25,321 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 01:50:25,323 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] RemoteTransactionContext - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 ReadData operation succeeded 01:50:25,323 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:50:25,323 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,323 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 4 01:50:25,323 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4 01:50:25,323 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:25,323 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 4 01:50:25,323 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 01:50:25,323 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry - waiting on 2 shards 01:50:25,323 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 01:50:25,323 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:50:25,323 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 01:50:25,323 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 01:50:25,323 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onShutDown 01:50:25,323 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onShutDown 01:50:25,323 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 01:50:25,323 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,323 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:50:25,324 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 01:50:25,324 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:25,324 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:50:25,324 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 01:50:25,324 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All shards shutdown - sending PoisonPill to self 01:50:25,324 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 01:50:25,325 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:25,325 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:50:25,325 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:25,325 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: shutting down 01:50:25,325 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:25,327 PM [cluster-test-akka.actor.default-dispatcher-17] [ERROR] OneForOneStrategy - Cannot invoke "org.opendaylight.controller.cluster.access.client.AbstractClientActorBehavior.persistenceId()" because "this.currentBehavior" is null java.lang.NullPointerException: Cannot invoke "org.opendaylight.controller.cluster.access.client.AbstractClientActorBehavior.persistenceId()" because "this.currentBehavior" is null at org.opendaylight.controller.cluster.access.client.AbstractClientActor.persistenceId(AbstractClientActor.java:31) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.onReceiveCommand(AbstractClientActor.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:25,330 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:25,330 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:25,334 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:25,335 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:25,335 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:25,335 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:25,337 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:25,357 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 01:50:25,360 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:50:25,360 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-492: Closing 01:50:25,360 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:50:25,360 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-491: Closing 01:50:25,360 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 01:50:25,360 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 01:50:25,361 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 01:50:25,362 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:25,363 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:26,347 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [1006403785873129215] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:26,378 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:26,379 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:26,379 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [7114834525764173596] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:26,380 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:26.381] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:26,401 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:26,402 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:26,407 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:26.409] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:26,424 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:50:26,430 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-542341603931610279] 01:50:26,431 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:26,432 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:26,432 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:26,444 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:26,444 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:26,444 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:26,444 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:26,444 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:26,447 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:26,453 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [26086138363950348] 01:50:26,453 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:26,454 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:26,454 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:26,458 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:26,458 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:26,477 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:26,494 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [6000894153967511337] 01:50:26,494 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:26,496 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:26,497 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:26,497 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:26,497 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:26,497 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:26,506 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:26,507 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:26,508 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,515 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,517 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:26,518 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1884506408] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:26,523 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 01:50:26,523 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,523 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:50:26,523 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,523 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@fdf5370 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1cd26754 01:50:26,523 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@26331f9c 01:50:26,523 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:26,523 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,524 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1710165026523, None)) 01:50:26,526 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165026522,0,0)) 01:50:26,527 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7a5ada63 01:50:26,527 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1cd26754 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@10bb63eb 01:50:26,527 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 01:50:26,527 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:26,528 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:26,528 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,528 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,528 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,528 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-2000903544] 01:50:26,529 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 01:50:26,529 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-2000903544] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:26,529 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 01:50:26,530 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-607278097] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:26,530 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:26,532 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,535 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-607278097 created and ready for shard:member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,536 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:26,536 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:26,536 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 01:50:26,536 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:50:26,536 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 01:50:26,536 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 01:50:26,536 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 01:50:26,536 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,537 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-1:snapshot-read#-2047046088] 01:50:26,537 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-1:snapshot-read#-2047046088] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:26,538 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 01:50:26,543 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,543 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#804961676] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:26,556 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:26,559 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:26,564 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 01:50:26,564 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:50:26,564 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2f44443b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6eab45a 01:50:26,564 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2d5205f 01:50:26,564 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1710165026564, None)) 01:50:26,564 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,564 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,564 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:26,564 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,564 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165026563,0,0)) 01:50:26,565 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@437025f4 01:50:26,565 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6eab45a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@378ca5df 01:50:26,565 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 01:50:26,565 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:26,565 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,565 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,566 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,566 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-745991389] 01:50:26,566 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 01:50:26,566 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-745991389] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:26,566 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-1227180698 created and ready for shard:member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,566 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-2:snapshot-read#-14959975] 01:50:26,566 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-1227180698] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:26,567 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-2:snapshot-read#-14959975] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:26,567 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 01:50:26,572 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:26,572 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:26,572 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 01:50:26,572 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:50:26,573 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 01:50:26,573 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 01:50:26,573 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 01:50:26,573 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,573 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,573 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 01:50:26,615 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1884506408] 01:50:26,616 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: findLocalShard : cars 01:50:26,616 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-2000903544] 01:50:26,832 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:26,832 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Received ElectionTimeout - switching to Candidate 01:50:26,832 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 01:50:26,832 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:26,832 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 1 01:50:26,832 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:26,832 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 01:50:26,832 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 01:50:26,832 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:50:26,832 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Follower to Candidate 01:50:26,833 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Follower to Candidate 01:50:27,121 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 01:50:27,121 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,121 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 2 01:50:27,126 PM [cluster-test-shard-dispatcher-19] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:27,382 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 01:50:27,382 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,382 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 3 01:50:27,463 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:27,463 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:27,463 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:27,463 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,463 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,463 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:27,463 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,463 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,463 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,463 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,480 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:27,480 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,480 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,480 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:27,480 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,480 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,480 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Peer address for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,611 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 01:50:27,611 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,611 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 4 01:50:27,611 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,612 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,615 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:27,615 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Cluster state: [] 01:50:27,615 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:27,615 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:27,615 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=null 01:50:27,615 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:27,615 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,615 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:27,615 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-2000903544] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:27,616 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:27,616 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:27,616 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Election: Leader has following peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 01:50:27,616 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:27,616 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:27,616 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:27,616 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 01:50:27,617 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: became leader with frontend state for [] 01:50:27,617 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Candidate to Leader 01:50:27,617 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 01:50:27,617 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:27,617 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:27,617 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Candidate to Leader 01:50:27,617 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 01:50:27,624 PM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:50:27,624 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,625 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 01:50:27,625 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:27,625 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-2000903544] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:27,625 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:27,625 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 01:50:27,625 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 01:50:27,626 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: -1, nextIndex: 0 01:50:27,626 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:27,627 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:27,628 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:27,630 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,634 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,634 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1703409995] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:27,646 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,646 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 01:50:27,646 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:50:27,646 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,646 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@cbdfbf8 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@213b34d9 01:50:27,646 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@41f01629 01:50:27,647 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:27,647 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,647 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:27,647 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,647 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:27,647 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,647 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1710165027647, None)) 01:50:27,648 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165027646,0,0)) 01:50:27,648 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@547519c9 01:50:27,648 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@213b34d9 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6f5bf4fb 01:50:27,648 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 01:50:27,649 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:27,649 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,649 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 01:50:27,649 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#804961676] 01:50:27,649 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 Queuing TransactionOperation 01:50:27,649 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,649 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,649 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-736942887] 01:50:27,649 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,649 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,649 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-736942887] 01:50:27,649 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,649 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,649 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-736942887] 01:50:27,649 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:27,649 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, primaryVersion=13] 01:50:27,649 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 01:50:27,650 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 01:50:27,650 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, primaryVersion=13] 01:50:27,650 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] for shard cars 01:50:27,650 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] found - trying create transaction 01:50:27,650 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] 01:50:27,651 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-736942887] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:27,651 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#278661857 created and ready for shard:member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,651 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-3:snapshot-read#1067565024] 01:50:27,651 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#278661857] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:27,652 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:27,652 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:27,652 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, disabling transaction tracking 01:50:27,652 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 01:50:27,652 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:50:27,652 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 01:50:27,652 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 01:50:27,652 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 01:50:27,652 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,652 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:27,652 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,652 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 01:50:27,652 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 01:50:27,652 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:27,652 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,652 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:27,652 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,653 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]]] 01:50:27,653 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating transaction : member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 01:50:27,653 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 01:50:27,653 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_18#1804229561] 01:50:27,653 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_18#1804229561] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:27,654 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 01:50:27,654 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=null 01:50:27,654 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:50:27,654 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,654 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 01:50:27,654 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:27,654 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:27,654 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 01:50:27,655 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: -1, nextIndex: 0 01:50:27,656 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:27,656 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,656 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]]] 01:50:27,656 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 01:50:27,656 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]]] 01:50:27,656 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 01:50:27,656 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:27,656 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 01:50:27,656 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 01:50:27,656 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 1 01:50:27,656 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:27,657 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_18#1804229561, transactionId=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0, version=13] 01:50:27,657 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:27,657 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_18#1804229561)] 01:50:27,657 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 01:50:27,657 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done true 01:50:27,658 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 0, nextIndex: 1 01:50:27,658 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:27,658 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Setting last applied to 0 01:50:27,658 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from -1 to 0 01:50:27,658 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 01:50:27,658 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:50:27,658 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} does not match any client, pre-disabling client. 01:50:27,658 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, nanosAgo=5766877} is already disabled 01:50:27,658 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:27,658 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=0 01:50:27,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]]] 01:50:27,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 01:50:27,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:27,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 01:50:27,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 01:50:27,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 1 01:50:27,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:27,658 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 01:50:27,659 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 0, nextIndex: 1 01:50:27,659 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:27,659 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 01:50:27,659 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done true 01:50:27,659 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Commit index set to 0 01:50:27,659 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:27,659 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Setting last applied to 0 01:50:27,659 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from -1 to 0 01:50:27,659 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 01:50:27,659 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:50:27,659 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} does not match any client, pre-disabling client. 01:50:27,660 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=0 01:50:27,660 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Commit index set to 0 01:50:27,661 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:27,661 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Setting last applied to 0 01:50:27,661 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from -1 to 0 01:50:27,661 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 01:50:27,661 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:50:27,661 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} does not match any client, pre-disabling client. 01:50:27,661 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=0 01:50:27,662 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-3:snapshot-read#1067565024] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:27,665 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 ReadData operation succeeded 01:50:27,665 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:50:27,666 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,666 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader)_slicer-id-497: Closing 01:50:27,666 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 01:50:27,666 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,666 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:27,666 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:27,669 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:27,669 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:27,670 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:27,670 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:27,671 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:27,693 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,694 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionRetryWithInitialAskTimeoutExOnCreateTx commit rateLimit adjusted to 162.12298098119095 01:50:27,694 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:50:27,694 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:27,694 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:27,694 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] found - trying create transaction 01:50:27,694 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] 01:50:27,694 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Queuing TransactionOperation 01:50:27,694 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Readying 1 components for commit 01:50:27,694 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Readying transaction for shard cars 01:50:27,694 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Queuing TransactionOperation 01:50:27,694 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 canCommit 01:50:27,695 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 01:50:27,701 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,701 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 01:50:27,701 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:27,701 PM [cluster-test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 01:50:27,701 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : true 01:50:27,701 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:50:28,206 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)]] after [500 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 01:50:28,235 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 01:50:28,236 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] found - trying create transaction 01:50:28,236 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] 01:50:28,266 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 01:50:28,266 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 01:50:28,266 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 01:50:28,266 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Received ElectionTimeout - switching to Candidate 01:50:28,266 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 01:50:28,266 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:28,266 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 5 01:50:28,266 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=4] to peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:28,267 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:28,267 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:50:28,267 PM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 01:50:28,267 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 01:50:28,267 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:50:28,267 PM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:50:28,267 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Follower to Candidate 01:50:28,267 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=null, leaderPayloadVersion=-1] 01:50:28,267 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:28,267 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:28,267 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Follower to Candidate 01:50:28,268 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:28,268 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Cluster state: [] 01:50:28,268 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:28,268 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 4 - updating term 01:50:28,268 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=null 01:50:28,268 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 0, lastTerm: 4 01:50:28,268 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:28,268 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:50:28,270 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:50:28,270 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:50:28,270 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Election: Leader has following peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 01:50:28,270 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:28,270 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:28,270 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:50:28,270 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 01:50:28,271 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: became leader with frontend state for [member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 01:50:28,271 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Candidate to Leader 01:50:28,271 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 01:50:28,271 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:28,271 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:28,271 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Candidate to Leader 01:50:28,271 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 01:50:28,272 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:50:28,272 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:28,272 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 01:50:28,272 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:28,272 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:28,272 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 01:50:28,272 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 01:50:28,272 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 0, nextIndex: 1 01:50:28,273 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:28,273 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done true 01:50:28,537 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:28,686 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-542341603931610279] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:28,686 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-542341603931610279] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:28,756 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)]] after [500 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 01:50:28,785 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 01:50:28,785 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#804961676] 01:50:28,786 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:28,786 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-745991389, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@3f286724] 01:50:28,786 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-745991389, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@3f286724] 01:50:28,787 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-745991389)] found - trying create transaction 01:50:28,787 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-745991389)] 01:50:28,787 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating transaction : member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:50:28,787 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:1_19#-35977124] 01:50:28,787 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:1_19#-35977124] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:28,787 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:1_19#-35977124, transactionId=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0, version=13] 01:50:28,788 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:28,788 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 directCommit called 01:50:28,788 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 sending 1 batched modifications, ready: true 01:50:28,788 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:1_19#-35977124)] 01:50:28,788 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:50:28,789 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: handleForwardedReadyTransaction for member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:50:28,789 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Readying transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0, client version 13 01:50:28,789 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: readying transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:50:28,789 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 ready 01:50:28,789 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Validating transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:50:28,789 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 validated 01:50:28,789 PM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:28,789 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: canCommit for member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: success 01:50:28,789 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Committing transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:50:28,789 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Preparing transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:50:28,789 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 candidate ready 01:50:28,789 PM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: canCommit - candidate rootPath: / 01:50:28,790 PM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: canCommit - messages: [] 01:50:28,790 PM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 01:50:28,790 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 prepared 01:50:28,790 PM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:28,790 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Finishing commit for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:50:28,790 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Starting commit for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:50:28,790 PM [cluster-test-shard-dispatcher-19] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 191 01:50:28,790 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=1, term=5, payload=Simple{size=191}] 01:50:28,790 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=Simple{size=191}] 01:50:28,790 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Replicate message: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:28,790 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:28,790 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:28,790 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=Simple{size=191}]]] 01:50:28,790 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:28,791 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 submitted to persistence 01:50:28,791 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=Simple{size=191}] 01:50:28,792 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=Simple{size=191}]]] 01:50:28,792 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 01:50:28,792 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:28,792 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log Simple{size=191} 01:50:28,792 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=Simple{size=191}] 01:50:28,792 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 2 01:50:28,792 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=1, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:28,793 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=Simple{size=191}] 01:50:28,793 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 1, nextIndex: 2 01:50:28,793 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:28,793 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Setting last applied to 1 01:50:28,793 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from 0 to 1 01:50:28,793 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 1 data Simple{size=191} 01:50:28,793 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Resuming commit of transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:50:28,794 PM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: commit - successfulFromPrevious: [] 01:50:28,794 PM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 01:50:28,794 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_shard-cars-member-2%3Adatastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx%400%3A1_19%23-35977124$d] 01:50:28,794 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#297520603] with name member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher 01:50:28,794 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:28,794 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=1 01:50:28,795 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 modifications completed with CommitTransactionReply [version=13] 01:50:28,795 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 successfully completed direct commit 01:50:28,795 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#297520603] 01:50:28,795 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: publishChanges: rootPath: / 01:50:28,795 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#297520603] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:28,795 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Elapsed time for generation of change events for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher: 92.86 μs 01:50:28,795 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:50:28,795 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Issuing gracefulStop to shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:28,795 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx - waiting on 1 shards 01:50:28,796 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onShutDown 01:50:28,796 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:28,796 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 01:50:28,796 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 01:50:28,796 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All shards shutdown - sending PoisonPill to self 01:50:28,796 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:28,796 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-745991389] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-736942887] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-736942887] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:28,796 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:50:28,796 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:50:28,796 PM [main] [WARN] ActorUtils - ShardManager for testTransactionRetryWithInitialAskTimeoutExOnCreateTx data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:50:28,797 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:50:28,797 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:50:28,797 PM [main] [WARN] ActorUtils - ShardManager for testTransactionRetryWithInitialAskTimeoutExOnCreateTx data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:50:28,797 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:50:28,796 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: shutting down 01:50:28,797 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:50:28,797 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#1313000487] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1313000487] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:28,797 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1703409995] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1703409995] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:28,797 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1703409995] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1703409995] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:28,797 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:50:28,797 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:28,797 PM [cluster-test-shard-dispatcher-18] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader)_slicer-id-500: Closing 01:50:28,798 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 01:50:28,798 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:50:28,798 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:28,798 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:29,817 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [26086138363950348] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:29,819 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:29,819 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:29,821 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:29.822] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:29,827 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:29,827 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:29,829 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:29.830] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:29,845 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:50:29,851 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4306200644667722288] 01:50:29,851 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:29,852 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:29,852 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:29,860 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:29,860 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:29,861 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:29,861 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:29,862 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:29,864 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:29,868 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [7309589368073628345] 01:50:29,869 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:29,871 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:29,871 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:29,871 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:29,871 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:29,882 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 closeTransaction called 01:50:29,882 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_16#-1162200257)] 01:50:29,882 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 closeTransaction called 01:50:29,882 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2557/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-2:datastore-testLeadershipTransferOnShutdown@0:2_14#148901206)] 01:50:29,882 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 closeTransaction called 01:50:29,882 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_18#1804229561)] [ERROR] [03/11/2024 13:50:29.883] [com.google.common.base.internal.Finalizer] [akka://cluster-test@127.0.0.1:2558/] swallowing exception during message send java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.remote.artery.Association.setupStopQuarantinedTimer(Association.scala:633) at akka.remote.artery.Association.send(Association.scala:399) at akka.remote.artery.ArteryTransport.send(ArteryTransport.scala:707) at akka.remote.RemoteActorRef.$bang(RemoteActorRefProvider.scala:737) at akka.actor.ActorRef.tell(ActorRef.scala:131) at akka.actor.ActorSelection$.rec$1(ActorSelection.scala:287) at akka.actor.ActorSelection$.deliverSelection(ActorSelection.scala:291) at akka.actor.ActorSelection.tell(ActorSelection.scala:50) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.sendOperationAsync(ActorUtils.java:398) at org.opendaylight.controller.cluster.datastore.RemoteTransactionContext.closeTransaction(RemoteTransactionContext.java:84) at org.opendaylight.controller.cluster.datastore.TransactionContextCleanup.finalizeReferent(TransactionContextCleanup.java:60) at jdk.internal.reflect.GeneratedMethodAccessor80.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at com.google.common.base.internal.Finalizer.finalizeReference(Finalizer.java:208) at com.google.common.base.internal.Finalizer.cleanUp(Finalizer.java:181) at com.google.common.base.internal.Finalizer.run(Finalizer.java:146) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:50:29,883 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 closeTransaction called 01:50:29,883 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-people-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_17#574230587)] 01:50:29,890 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:29,900 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [3031617293159600060] 01:50:29,900 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:29,902 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:29,902 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:29,905 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:29,905 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:29,913 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:29,915 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:29,916 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithMultipleShards 01:50:29,921 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 01:50:29,921 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-1077670983] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,924 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithMultipleShards 01:50:29,924 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:29,924 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: no initial generation override, starting from 0 01:50:29,924 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:50:29,924 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@49f1d936 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7c55f4e0 01:50:29,924 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4f995b3b 01:50:29,924 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:29,924 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:29,924 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:29,924 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:29,924 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, 0, 1710165029924, None)) 01:50:29,925 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165029923,0,0)) 01:50:29,925 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6f558ae9 01:50:29,925 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7c55f4e0 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@abda087 01:50:29,925 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} started 01:50:29,925 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:29,927 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:29,928 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithMultipleShards 01:50:29,929 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:29,929 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:29,929 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:29,929 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:29,929 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards on shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-2050818085] 01:50:29,929 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:29,929 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:29,929 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:29,929 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards on shard member-1-shard-people-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-465730837] 01:50:29,930 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Shard created : member-1-shard-cars-testReadWriteTransactionWithMultipleShards, persistent : true 01:50:29,930 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-2050818085] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,930 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-people-testReadWriteTransactionWithMultipleShards, persistent : true 01:50:29,930 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-465730837] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,931 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#-101627955 created and ready for shard:member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:29,931 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#-101627955] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,931 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#180392359] 01:50:29,931 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-people-testReadWriteTransactionWithMultipleShards with journal batch size 1 01:50:29,931 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-notifier#838326523 created and ready for shard:member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:29,931 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#878044807] 01:50:29,931 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#180392359] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,931 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadWriteTransactionWithMultipleShards with journal batch size 1 01:50:29,932 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:29,932 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:29,932 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:50:29,932 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 01:50:29,932 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 01:50:29,932 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [people] 01:50:29,932 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is initialized 01:50:29,932 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:29,932 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 01:50:29,932 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:29,932 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from null to Follower 01:50:29,932 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [cars] 01:50:29,932 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is initialized 01:50:29,932 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:50:29,932 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 01:50:29,932 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 01:50:29,932 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 01:50:29,932 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from null to Follower 01:50:29,934 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-notifier#838326523] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,934 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#878044807] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,939 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-1423044841] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,943 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: no initial generation override, starting from 0 01:50:29,943 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:50:29,943 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5d1016c9 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@53dc3ad7 01:50:29,943 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@79b8e3a9 01:50:29,943 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, 0, 1710165029943, None)) 01:50:29,943 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165029942,0,0)) 01:50:29,944 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@53dc3ad7 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6b5a8351 01:50:29,944 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} started 01:50:29,947 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 01:50:29,948 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithMultipleShards 01:50:29,948 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:29,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:29,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@18a377f0 01:50:29,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:29,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:29,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:29,950 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-cars-testReadWriteTransactionWithMultipleShards, persistent : true 01:50:29,950 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-people-testReadWriteTransactionWithMultipleShards, persistent : true 01:50:29,950 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadWriteTransactionWithMultipleShards with journal batch size 1 01:50:29,950 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-people-testReadWriteTransactionWithMultipleShards with journal batch size 1 01:50:29,950 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards#-1439078376] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,951 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards#-411750189] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,951 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#92384793 created and ready for shard:member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:29,951 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#92384793] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,951 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-977498392] 01:50:29,951 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-2102417522] 01:50:29,951 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-1233118418] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,951 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:29,951 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:29,951 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:29,951 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:29,952 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:50:29,952 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 01:50:29,952 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [cars] 01:50:29,952 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards is initialized 01:50:29,952 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [people] 01:50:29,952 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards is initialized 01:50:29,953 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-1233118418 created and ready for shard:member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:29,953 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 01:50:29,953 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-977498392] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,953 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-2102417522] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:29,954 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:50:29,954 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 01:50:29,954 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 01:50:29,954 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 01:50:29,954 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 01:50:29,954 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-2-shard-people-testReadWriteTransactionWithMultipleShards from null to Follower 01:50:29,954 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-2-shard-cars-testReadWriteTransactionWithMultipleShards from null to Follower 01:50:29,963 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:29,967 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:29,969 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:29,969 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:29,969 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:29,969 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:29,969 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:29,969 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:29,969 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:29,972 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:29,973 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:29,994 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-1077670983] 01:50:29,994 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: findLocalShard : cars 01:50:29,995 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-2050818085] 01:50:30,181 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:30,181 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:50:30,181 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadWriteTransactionWithMultipleShards] 01:50:30,181 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:30,181 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 01:50:30,181 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:30,181 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Closing 01:50:30,181 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:50:30,181 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 01:50:30,181 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from Follower to Candidate 01:50:30,182 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from Follower to Candidate 01:50:30,220 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:30,220 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:50:30,220 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testReadWriteTransactionWithMultipleShards] 01:50:30,220 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:30,220 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 01:50:30,220 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:30,221 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Closing 01:50:30,221 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:50:30,221 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 01:50:30,221 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from Follower to Candidate 01:50:30,222 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from Follower to Candidate 01:50:30,410 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:50:30,410 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:30,410 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 01:50:30,451 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:50:30,451 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:30,451 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 01:50:30,484 PM [cluster-test-shard-dispatcher-20] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:30,553 PM [cluster-test-shard-dispatcher-20] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:30,731 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:50:30,731 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:30,731 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 01:50:30,751 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:50:30,751 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:30,751 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 01:50:30,872 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:30,872 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:30,873 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:30,873 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:30,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:30,873 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards 01:50:30,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:30,873 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:30,873 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:30,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:30,873 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:30,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:30,873 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:30,873 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:30,883 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:30,883 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:30,883 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:30,883 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards 01:50:30,883 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:30,883 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:30,883 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:30,883 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:30,883 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:30,883 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:30,918 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:30,990 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:50:30,990 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:30,990 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 01:50:30,990 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:30,991 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:30,992 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Cluster state: [] 01:50:30,992 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:30,992 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:30,992 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 01:50:30,992 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:30,992 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:30,992 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:30,998 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:30,998 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:30,998 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testReadWriteTransactionWithMultipleShards] 01:50:30,998 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:30,998 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:30,998 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:50:30,998 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: became leader with frontend state for [] 01:50:30,998 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from Candidate to Leader 01:50:30,998 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:50:30,998 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:30,998 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:30,998 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from Candidate to Leader 01:50:30,999 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Closing 01:50:30,999 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testReadWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:30,999 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:50:30,999 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:30,999 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:30,999 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testReadWriteTransactionWithMultipleShards status sync done false 01:50:31,0 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:50:31,0 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:31,51 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:50:31,51 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:31,51 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 01:50:31,51 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:31,52 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:31,52 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Cluster state: [] 01:50:31,52 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:31,52 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:31,52 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 01:50:31,52 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:31,52 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:31,52 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:31,54 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:31,54 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:31,54 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadWriteTransactionWithMultipleShards] 01:50:31,54 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:31,54 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:31,54 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:50:31,54 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: became leader with frontend state for [] 01:50:31,54 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from Candidate to Leader 01:50:31,54 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:50:31,54 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:31,54 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:31,54 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from Candidate to Leader 01:50:31,54 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All Shards are ready - data store testReadWriteTransactionWithMultipleShards is ready 01:50:31,55 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Closing 01:50:31,55 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testReadWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:31,55 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:50:31,55 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:31,55 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:31,55 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All Shards are ready - data store testReadWriteTransactionWithMultipleShards is ready 01:50:31,55 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithMultipleShards status sync done false 01:50:31,56 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:50:31,56 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:31,111 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testReadWriteTransactionWithMultipleShards status sync done true 01:50:31,152 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-1077670983] 01:50:31,152 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: findLocalShard : people 01:50:31,152 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-465730837] 01:50:31,153 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testReadWriteTransactionWithMultipleShards commit rateLimit adjusted to 162.12298098119095 01:50:31,153 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,153 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-1423044841] 01:50:31,153 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 01:50:31,153 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-1423044841] 01:50:31,153 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 01:50:31,153 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying 2 components for commit 01:50:31,153 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying transaction for shard cars 01:50:31,153 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 01:50:31,153 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying transaction for shard people 01:50:31,153 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 01:50:31,153 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 canCommit 01:50:31,154 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:31,154 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 01:50:31,154 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:50:31,154 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 01:50:31,154 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 01:50:31,154 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 01:50:31,154 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] for shard cars 01:50:31,154 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] for shard people 01:50:31,154 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] found - trying create transaction 01:50:31,154 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] found - trying create transaction 01:50:31,154 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] 01:50:31,154 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] 01:50:31,156 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, disabling transaction tracking 01:50:31,156 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, disabling transaction tracking 01:50:31,156 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 01:50:31,156 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 01:50:31,156 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 01:50:31,156 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 01:50:31,156 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:31,156 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:31,156 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:31,156 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:31,156 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:31,156 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}]]] 01:50:31,156 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:31,156 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,156 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}]]] 01:50:31,156 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,156 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_20#-1078504652] 01:50:31,157 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_21#-1691626426] 01:50:31,157 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_20#-1078504652] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:31,157 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_21#-1691626426] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:31,157 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 01:50:31,157 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 01:50:31,158 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}]]] 01:50:31,158 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:50:31,158 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:31,158 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 01:50:31,158 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 01:50:31,158 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 1 01:50:31,158 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:31,159 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}]]] 01:50:31,159 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:50:31,159 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:31,159 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 01:50:31,159 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 01:50:31,159 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 1 01:50:31,159 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:31,160 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:50:31,160 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:50:31,160 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:31,160 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:31,160 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 01:50:31,160 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:50:31,160 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 01:50:31,160 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:50:31,160 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 01:50:31,160 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:50:31,160 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 01:50:31,160 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-people-testReadWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:50:31,160 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:50:31,160 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, nanosAgo=3944381} is already disabled 01:50:31,160 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:50:31,160 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, nanosAgo=3992692} is already disabled 01:50:31,160 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:31,160 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:31,160 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:50:31,160 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:50:31,161 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_20#-1078504652, transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] 01:50:31,161 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_21#-1691626426, transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] 01:50:31,161 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:31,161 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 01:50:31,161 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 = [ 1 ] 01:50:31,161 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction called 01:50:31,161 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 = [ 1 ] 01:50:31,161 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction called 01:50:31,161 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 sending 1 batched modifications, ready: true 01:50:31,161 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 sending 1 batched modifications, ready: true 01:50:31,161 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_21#-1691626426)] 01:50:31,161 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_20#-1078504652)] 01:50:31,162 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 01:50:31,162 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 0 01:50:31,162 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:31,162 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 01:50:31,162 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:50:31,162 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 01:50:31,162 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:50:31,162 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:50:31,162 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 01:50:31,162 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithMultipleShards status sync done true 01:50:31,162 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:50:31,162 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 0 01:50:31,162 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:31,162 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 01:50:31,162 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:50:31,162 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 01:50:31,162 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:50:31,162 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-people-testReadWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:50:31,162 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:50:31,163 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,163 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: handleForwardedReadyTransaction for member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,163 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, client version 13 01:50:31,163 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,163 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,163 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 ready 01:50:31,163 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: handleForwardedReadyTransaction for member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,163 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, client version 13 01:50:31,163 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,163 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 ready 01:50:31,164 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction 01:50:31,164 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-465730837)] resolved 01:50:31,164 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:50:31,164 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction 01:50:31,164 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-2050818085)] resolved 01:50:31,164 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: successfully resolved all cohort actors 01:50:31,164 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 finishCanCommit 01:50:31,164 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-2050818085)] 01:50:31,164 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-2050818085)] 01:50:31,164 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:50:31,165 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Can committing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,165 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,165 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 validated 01:50:31,165 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:31,165 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: canCommit for member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: success 01:50:31,166 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:50:31,166 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-465730837)] 01:50:31,166 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-465730837)] 01:50:31,167 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Can committing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,167 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,167 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 validated 01:50:31,167 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:31,167 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: canCommit for member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: success 01:50:31,168 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:50:31,168 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit returning result: true 01:50:31,168 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 commit 01:50:31,168 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 finish commit 01:50:31,168 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-2050818085)] 01:50:31,168 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-2050818085)] 01:50:31,168 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-465730837)] 01:50:31,168 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-465730837)] 01:50:31,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Committing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 candidate ready 01:50:31,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:50:31,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Committing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - messages: [] 01:50:31,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:50:31,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 prepared 01:50:31,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:31,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Finishing commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 candidate ready 01:50:31,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:50:31,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 179 01:50:31,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - messages: [] 01:50:31,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:50:31,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:50:31,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:50:31,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 prepared 01:50:31,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:31,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Finishing commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:31,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:31,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:31,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 183 01:50:31,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 01:50:31,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 01:50:31,169 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 submitted to persistence 01:50:31,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 01:50:31,170 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:31,170 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:31,170 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:31,170 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}]]] 01:50:31,170 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 submitted to persistence 01:50:31,170 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:50:31,170 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 01:50:31,171 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 01:50:31,171 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}]]] 01:50:31,171 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:50:31,171 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:31,171 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:50:31,171 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=179} 01:50:31,171 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:31,171 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:50:31,171 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=183} 01:50:31,171 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 01:50:31,171 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 2 01:50:31,171 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 2 01:50:31,171 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:31,171 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:31,171 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 01:50:31,171 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 01:50:31,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:50:31,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:31,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 01:50:31,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:50:31,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=183} 01:50:31,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:50:31,172 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:50:31,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:50:31,172 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:31,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testReadWriteTransactionWithMultipleShards_member-1-shard-people-testReadWriteTransactionWithMultipleShards%23-465730837$h] 01:50:31,172 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 01:50:31,172 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:50:31,172 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=179} 01:50:31,172 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,172 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:50:31,172 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:50:31,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1936020811] with name member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher 01:50:31,172 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testReadWriteTransactionWithMultipleShards_member-1-shard-cars-testReadWriteTransactionWithMultipleShards%23-2050818085$g] 01:50:31,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:50:31,172 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1828933401] with name member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher 01:50:31,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:31,172 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:50:31,172 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1936020811] 01:50:31,173 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:31,173 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:50:31,173 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1828933401] 01:50:31,173 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:50:31,173 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:50:31,173 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 90.68 μs 01:50:31,173 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 104.6 μs 01:50:31,173 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1936020811] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:31,173 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:50:31,174 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit succeeded 01:50:31,174 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:50:31,175 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:31,175 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:31,175 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] found - trying create transaction 01:50:31,175 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] 01:50:31,175 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Queuing TransactionOperation 01:50:31,175 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 1 01:50:31,175 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:31,175 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 01:50:31,175 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:50:31,175 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=183} 01:50:31,175 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 1 01:50:31,175 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,175 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:31,175 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 01:50:31,175 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:50:31,175 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=179} 01:50:31,175 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:50:31,175 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-173795663] with name member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher 01:50:31,175 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:50:31,176 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:50:31,176 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-173795663] 01:50:31,176 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#142153589] with name member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher 01:50:31,176 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:50:31,176 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:50:31,176 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:50:31,176 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-173795663] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:31,176 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 56.18 μs 01:50:31,176 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#142153589] 01:50:31,176 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:50:31,176 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#142153589] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:31,176 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 60.78 μs 01:50:31,177 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:50:31,177 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_22#-962814835] 01:50:31,178 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_22#-962814835, transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0, version=13] 01:50:31,178 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:31,178 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_22#-962814835)] 01:50:31,179 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 ReadData operation succeeded 01:50:31,179 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] for shard people, trying to use DirectTransactionContextWrapper 01:50:31,179 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:31,179 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] found - trying create transaction 01:50:31,179 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] 01:50:31,179 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Queuing TransactionOperation 01:50:31,180 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:50:31,180 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_23#-1386525328] 01:50:31,180 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_23#-1386525328, transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0, version=13] 01:50:31,180 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 01:50:31,180 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_23#-1386525328)] 01:50:31,182 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 ReadData operation succeeded 01:50:31,182 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:50:31,182 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:31,182 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:31,182 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithMultipleShards - waiting on 2 shards 01:50:31,182 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onShutDown 01:50:31,182 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Initiating leader transfer 01:50:31,182 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onShutDown 01:50:31,182 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Initiating leader transfer 01:50:31,182 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@462797f 01:50:31,182 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6a78d787 01:50:31,182 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@462797f 01:50:31,182 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6a78d787 01:50:31,182 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 01:50:31,182 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 01:50:31,182 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:31,182 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:31,182 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:50:31,182 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:31,182 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:31,182 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:50:31,182 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:31,182 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:31,183 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards] 01:50:31,183 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:50:31,183 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:50:31,183 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:50:31,183 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards] 01:50:31,183 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:50:31,183 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:31,183 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:31,184 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:50:31,184 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testReadWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:50:31,184 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:50:31,184 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:50:31,184 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:31,184 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:31,184 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Leader transfer complete 01:50:31,184 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Leader transfer complete 01:50:31,601 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 01:50:31,601 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-2050818085]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:31,601 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:31,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader)_slicer-id-508: Closing 01:50:31,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:50:31,601 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:50:31,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader)_slicer-id-507: Closing 01:50:31,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:50:31,601 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 01:50:31,601 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 01:50:31,601 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:50:31,602 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 01:50:31,602 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 01:50:31,603 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:31,603 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:31,607 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:31,608 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:31,608 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:31,608 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:31,622 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 01:50:31,622 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:50:31,622 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Closing 01:50:31,622 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:50:31,622 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:50:31,622 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Closing 01:50:31,622 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:50:31,623 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 01:50:31,623 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:31,623 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:32,617 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4306200644667722288] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:32,635 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:32,636 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:32,638 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [7309589368073628345] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [03/11/2024 13:50:32.638] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:32,645 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:32,645 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:32,647 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:32.648] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:32,662 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:32,669 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [8810265178323600573] 01:50:32,669 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:32,674 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:32,674 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:32,678 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:32,678 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:32,688 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:32,688 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:32,688 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:32,691 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:32,707 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1020357087298024118] 01:50:32,707 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:32,714 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:32,714 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:32,723 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:32,723 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:32,746 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:32,757 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [3099567158063002066] 01:50:32,757 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:32,762 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:32,762 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:32,769 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:32,769 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:32,770 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:32,770 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:32,780 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:32,783 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:32,788 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:32,788 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:32,790 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:32,793 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 01:50:32,794 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 01:50:32,803 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:32,804 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:33,822 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:33.823] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:33,840 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:33,840 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:33,843 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:33.843] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:33,860 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:33,866 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-6490653483033289381] 01:50:33,866 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:33,870 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:33,870 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:33,870 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:33,870 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:33,880 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:33,880 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:33,880 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:33,884 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:33,896 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-8431985092865729004] 01:50:33,896 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:33,903 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:33,903 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:33,912 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:33,912 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:33,920 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:50:33,929 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [8601440369962078032] 01:50:33,929 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:33,935 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:33,935 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:33,935 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:33,935 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:33,942 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:33,943 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:33,945 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleTransactionsWritesInQuickSuccession 01:50:33,960 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:33,961 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:33,975 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-1222322845] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:33,976 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 01:50:33,992 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testSingleTransactionsWritesInQuickSuccession 01:50:33,992 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Creating local shard: member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:33,993 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:33,993 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:33,999 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: no initial generation override, starting from 0 01:50:33,999 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:50:33,999 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7fb4a9b0 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5db36d0a 01:50:33,999 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@598cf264 01:50:33,999 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, 0, 1710165033999, None)) 01:50:34,4 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165033998,0,0)) 01:50:34,4 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5db36d0a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2ad65721 01:50:34,4 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@16e9d7bf 01:50:34,4 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} started 01:50:34,5 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:34,5 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,5 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,5 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,5 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: updated peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1108744531] 01:50:34,5 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, persistent : true 01:50:34,5 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1108744531] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:34,5 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:34,7 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:34,8 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleTransactionsWritesInQuickSuccession 01:50:34,10 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with journal batch size 1 01:50:34,10 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-1:snapshot-read#-1127180142] 01:50:34,10 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#-1873962707 created and ready for shard:member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,10 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-1:snapshot-read#-1127180142] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:34,10 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#-1873962707] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:34,12 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:34,12 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:34,12 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 01:50:34,12 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 01:50:34,12 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Initializing shard [cars] 01:50:34,12 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is initialized 01:50:34,12 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from null to Follower 01:50:34,12 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , registered listener akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession 01:50:34,12 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from null to Follower 01:50:34,13 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 01:50:34,13 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#535511149] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:34,15 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testSingleTransactionsWritesInQuickSuccession 01:50:34,15 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: no initial generation override, starting from 0 01:50:34,15 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:50:34,15 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Creating local shard: member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,15 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@52f0a050 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@230dc0a0 01:50:34,15 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@76e4f935 01:50:34,15 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, 0, 1710165034015, None)) 01:50:34,15 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165034014,0,0)) 01:50:34,15 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:34,15 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,15 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@468caa1c 01:50:34,15 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@230dc0a0 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@49a6ce7a 01:50:34,15 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} started 01:50:34,16 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:34,16 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,16 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,16 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,16 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: updated peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession#656408128] 01:50:34,17 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, persistent : true 01:50:34,17 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with journal batch size 1 01:50:34,18 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#90575494 created and ready for shard:member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,18 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-2:snapshot-read#-1027214544] 01:50:34,18 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession#656408128] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:34,18 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#90575494] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:34,18 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-2:snapshot-read#-1027214544] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:34,19 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:34,19 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:34,19 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 01:50:34,19 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 01:50:34,19 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,20 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Initializing shard [cars] 01:50:34,20 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession is initialized 01:50:34,20 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from null to Follower 01:50:34,21 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:34,21 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:34,22 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession , registered listener akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession 01:50:34,24 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession from null to Follower 01:50:34,66 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-1222322845] 01:50:34,66 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: findLocalShard : cars 01:50:34,66 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1108744531] 01:50:34,249 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:34,249 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Received ElectionTimeout - switching to Candidate 01:50:34,249 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] 01:50:34,249 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,249 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 1 01:50:34,249 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:34,249 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Closing 01:50:34,249 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 01:50:34,249 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 01:50:34,250 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from Follower to Candidate 01:50:34,250 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from Follower to Candidate 01:50:34,479 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 01:50:34,479 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,479 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 2 01:50:34,560 PM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:34,759 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 01:50:34,759 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,759 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 3 01:50:34,880 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:34,880 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:34,881 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:34,881 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,881 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,881 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:34,881 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,881 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,881 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,889 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:34,889 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,889 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,889 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:34,889 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,889 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:34,946 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:35,29 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 01:50:35,29 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:35,29 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 4 01:50:35,29 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:35,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:35,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Cluster state: [] 01:50:35,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:35,32 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:35,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=null 01:50:35,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:35,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:35,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:35,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:35,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:35,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] 01:50:35,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:35,34 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:35,34 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: true, hasLeader : true 01:50:35,34 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: became leader with frontend state for [] 01:50:35,34 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderPayloadVersion=13] 01:50:35,34 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:35,34 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:35,34 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from Candidate to Leader 01:50:35,34 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from Candidate to Leader 01:50:35,34 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All Shards are ready - data store testSingleTransactionsWritesInQuickSuccession is ready 01:50:35,35 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Closing 01:50:35,35 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: getLeaderAddress leaderId = member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:35,35 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession Received follower initial sync status for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession status sync done false 01:50:35,35 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderPayloadVersion=13] 01:50:35,35 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:35,35 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:35,35 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All Shards are ready - data store testSingleTransactionsWritesInQuickSuccession is ready 01:50:35,36 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: -1, nextIndex: 0 01:50:35,36 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:35,72 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testSingleTransactionsWritesInQuickSuccession commit rateLimit adjusted to 162.12298098119095 01:50:35,72 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:50:35,72 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#535511149] 01:50:35,72 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:50:35,72 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:50:35,72 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Readying 1 components for commit 01:50:35,72 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 01:50:35,72 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:50:35,72 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:35,72 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 canCommit 01:50:35,72 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, primaryVersion=13] 01:50:35,73 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, primaryVersion=13] 01:50:35,73 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars 01:50:35,73 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 01:50:35,73 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:35,73 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:50:35,73 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 directCommit called 01:50:35,73 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 sending 2 batched modifications, ready: true 01:50:35,73 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:50:35,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, disabling transaction tracking 01:50:35,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 01:50:35,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 01:50:35,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:35,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:35,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:35,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}]]] 01:50:35,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 01:50:35,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 01:50:35,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:50:35,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:50:35,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6339956 01:50:35,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 2 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 of 2 operations, client version 13 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 ready 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 validated 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: success 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 candidate ready 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: canCommit - candidate rootPath: / 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: canCommit - messages: [] 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: preCommit - successfulFromPrevious: [] 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 prepared 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 183 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 01:50:35,76 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:35,76 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}]]] 01:50:35,77 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:50:35,77 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:35,77 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:35,77 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:35,77 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72} 01:50:35,77 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 submitted to persistence 01:50:35,77 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 01:50:35,77 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 1 01:50:35,77 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:35,78 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 01:50:35,78 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession Received follower initial sync status for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession status sync done true 01:50:35,78 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 01:50:35,78 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 01:50:35,78 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 0, nextIndex: 1 01:50:35,78 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:35,78 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 0 01:50:35,78 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from -1 to 0 01:50:35,78 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72} 01:50:35,78 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:50:35,78 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} does not match any client, pre-disabling client. 01:50:35,78 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, nanosAgo=3786917} is already disabled 01:50:35,79 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:35,79 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:35,79 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}]]] 01:50:35,79 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 01:50:35,79 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=0 01:50:35,80 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}]]] 01:50:35,80 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 01:50:35,80 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:35,80 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74} 01:50:35,80 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 01:50:35,80 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=183} 01:50:35,80 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 01:50:35,80 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 01:50:35,80 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 0 01:50:35,80 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:35,80 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 2 01:50:35,80 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 0 01:50:35,80 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from -1 to 0 01:50:35,80 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72} 01:50:35,80 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:50:35,80 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} does not match any client, pre-disabling client. 01:50:35,81 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 01:50:35,81 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 01:50:35,81 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=0 01:50:35,86 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 2, nextIndex: 3 01:50:35,86 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 01:50:35,86 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 1 01:50:35,86 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 0 to 1 01:50:35,86 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74} 01:50:35,86 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} has no callback 01:50:35,86 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 2 01:50:35,86 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 1 to 2 01:50:35,86 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 2 data Simple{size=183} 01:50:35,86 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:50:35,86 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: commit - successfulFromPrevious: [] 01:50:35,86 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:50:35,86 PM [cluster-test-shard-dispatcher-28] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 01:50:35,86 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$a] 01:50:35,86 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#1921962303] with name member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher 01:50:35,86 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 1 for term 4 01:50:35,87 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:35,87 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=2 01:50:35,87 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#1921962303] 01:50:35,87 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:35,87 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 87.45 μs 01:50:35,87 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#1921962303] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:35,89 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 successfully completed direct commit 01:50:35,89 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 2 01:50:35,89 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 modifications completed with CommitTransactionReply [version=13] 01:50:35,89 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 01:50:35,89 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 1 01:50:35,89 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 0 to 1 01:50:35,89 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74} 01:50:35,89 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:50:35,89 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 2 01:50:35,89 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 1 to 2 01:50:35,89 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 2 data Simple{size=183} 01:50:35,89 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:50:35,90 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:35,90 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:35,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-192362364] with name member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher 01:50:35,90 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 01:50:35,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 1 for term 4 01:50:35,90 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-192362364] 01:50:35,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=2 01:50:35,90 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}] 01:50:35,90 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 Readying 1 components for commit 01:50:35,90 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 Readying transaction for shard cars 01:50:35,90 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 directCommit called 01:50:35,90 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 sending 1 batched modifications, ready: true 01:50:35,90 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:50:35,90 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:35,90 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 canCommit 01:50:35,90 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-192362364] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:35,90 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 135.1 μs 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@18ac78f8 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 of 1 operations, client version 13 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 ready 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}], operation=WRITE}}}}}}}]} 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 validated 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: success 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 candidate ready 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: canCommit - candidate rootPath: / 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: canCommit - messages: [] 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: preCommit - successfulFromPrevious: [] 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 prepared 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 01:50:35,92 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 01:50:35,93 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 01:50:35,93 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:35,93 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 01:50:35,93 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:35,93 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}]]] 01:50:35,93 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 submitted to persistence 01:50:35,94 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}]]] 01:50:35,94 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:50:35,94 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:50:35,94 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 01:50:35,94 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 01:50:35,94 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 01:50:35,94 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:35,94 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 01:50:35,97 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 3, nextIndex: 4 01:50:35,97 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 1 01:50:35,97 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 01:50:35,97 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 01:50:35,97 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 3 01:50:35,97 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 2 to 3 01:50:35,97 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 3 data Simple{size=240} 01:50:35,97 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:50:35,97 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: commit - successfulFromPrevious: [] 01:50:35,97 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 01:50:35,97 PM [cluster-test-shard-dispatcher-28] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:50:35,97 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$b] 01:50:35,97 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:35,97 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 2 for term 4 01:50:35,97 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=3 01:50:35,97 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 102.5 μs 01:50:35,98 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 modifications completed with CommitTransactionReply [version=13] 01:50:35,98 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 successfully completed direct commit 01:50:35,99 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 01:50:35,99 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:35,99 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:35,99 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 01:50:35,99 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:50:35,99 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 Queuing TransactionOperation 01:50:35,100 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 01:50:35,100 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:50:35,100 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@3d477e36 01:50:35,100 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_24#497585076] 01:50:35,100 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_24#497585076] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:35,101 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_24#497585076, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0, version=13] 01:50:35,101 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:35,101 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_24#497585076)] 01:50:35,103 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 ReadData operation succeeded 01:50:35,103 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 closeTransaction called 01:50:35,103 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_24#497585076)] 01:50:35,103 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:50:35,103 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:35,103 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:35,103 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 01:50:35,103 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}] 01:50:35,103 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 Readying 1 components for commit 01:50:35,104 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 Readying transaction for shard cars 01:50:35,104 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 directCommit called 01:50:35,104 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 sending 1 batched modifications, ready: true 01:50:35,104 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:50:35,104 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 canCommit 01:50:35,105 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:50:35,105 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@3d477e36 01:50:35,105 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:50:35,105 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 of 1 operations, client version 13 01:50:35,105 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:50:35,105 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 ready 01:50:35,106 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}], operation=WRITE}}}}}}}]} 01:50:35,106 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:50:35,106 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 validated 01:50:35,106 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:35,106 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: success 01:50:35,106 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:50:35,106 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:50:35,106 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 candidate ready 01:50:35,106 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: canCommit - candidate rootPath: / 01:50:35,106 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: canCommit - messages: [] 01:50:35,106 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: preCommit - successfulFromPrevious: [] 01:50:35,106 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 prepared 01:50:35,106 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:35,106 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:50:35,106 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:50:35,107 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 01:50:35,107 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 01:50:35,107 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 01:50:35,107 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:35,107 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 2 01:50:35,107 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:35,107 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}]]] 01:50:35,107 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 submitted to persistence 01:50:35,107 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 01:50:35,108 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}]]] 01:50:35,108 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:50:35,108 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 01:50:35,108 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 01:50:35,108 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 01:50:35,108 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 01:50:35,108 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 3 01:50:35,108 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:35,108 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 4 01:50:35,108 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 3 01:50:35,108 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 2 to 3 01:50:35,108 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 3 data Simple{size=240} 01:50:35,108 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:50:35,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 4, nextIndex: 5 01:50:35,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 01:50:35,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 4 01:50:35,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 3 to 4 01:50:35,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 4 data Simple{size=240} 01:50:35,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:50:35,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: commit - successfulFromPrevious: [] 01:50:35,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 changing state from COMMIT_PENDING to COMMITTED 01:50:35,109 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:35,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:50:35,109 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 2 for term 4 01:50:35,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$e] 01:50:35,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 3 for term 4 01:50:35,109 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 90.98 μs 01:50:35,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:50:35,109 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=4 01:50:35,109 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 01:50:35,109 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:35,109 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=3 01:50:35,109 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 40.60 μs 01:50:35,110 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 modifications completed with CommitTransactionReply [version=13] 01:50:35,110 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 successfully completed direct commit 01:50:35,110 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 01:50:35,110 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:35,110 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:35,110 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 01:50:35,110 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:50:35,111 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 Queuing TransactionOperation 01:50:35,111 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 01:50:35,111 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:50:35,111 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@69239d8d 01:50:35,111 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_25#1146895884] 01:50:35,112 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_25#1146895884] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:35,112 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_25#1146895884, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0, version=13] 01:50:35,112 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:35,112 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_25#1146895884)] 01:50:35,114 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 ReadData operation succeeded 01:50:35,114 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 closeTransaction called 01:50:35,114 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_25#1146895884)] 01:50:35,114 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:50:35,114 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:35,114 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:35,114 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 01:50:35,114 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}] 01:50:35,114 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 Readying 1 components for commit 01:50:35,114 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 Readying transaction for shard cars 01:50:35,114 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 directCommit called 01:50:35,114 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 sending 1 batched modifications, ready: true 01:50:35,114 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:50:35,114 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 canCommit 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@69239d8d 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 of 1 operations, client version 13 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 ready 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}], operation=WRITE}}}}}}}]} 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 validated 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: success 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 candidate ready 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: canCommit - candidate rootPath: / 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: canCommit - messages: [] 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: preCommit - successfulFromPrevious: [] 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 prepared 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 3 01:50:35,116 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:35,117 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}]]] 01:50:35,117 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 submitted to persistence 01:50:35,117 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 01:50:35,118 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 4 01:50:35,118 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4 01:50:35,118 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 4 01:50:35,118 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 3 to 4 01:50:35,118 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 4 data Simple{size=240} 01:50:35,118 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:50:35,119 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 3 for term 4 01:50:35,119 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:35,119 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=4 01:50:35,119 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 92.91 μs 01:50:35,119 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}]]] 01:50:35,119 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:50:35,119 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 01:50:35,119 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 01:50:35,119 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 01:50:35,119 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 01:50:35,119 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=5, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:35,120 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:35,120 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 5, nextIndex: 6 01:50:35,120 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 01:50:35,120 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 5 01:50:35,120 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 4 to 5 01:50:35,120 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 5 data Simple{size=240} 01:50:35,120 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:50:35,120 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: commit - successfulFromPrevious: [] 01:50:35,120 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 changing state from COMMIT_PENDING to COMMITTED 01:50:35,120 PM [cluster-test-shard-dispatcher-28] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:50:35,120 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$h] 01:50:35,121 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 4 for term 4 01:50:35,121 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 01:50:35,121 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=5 01:50:35,121 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:35,121 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 75.94 μs 01:50:35,127 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 01:50:35,127 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 5 01:50:35,127 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 successfully completed direct commit 01:50:35,127 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5 01:50:35,127 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 5 01:50:35,127 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 4 to 5 01:50:35,127 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 modifications completed with CommitTransactionReply [version=13] 01:50:35,127 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 5 data Simple{size=240} 01:50:35,127 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:50:35,128 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 4 for term 4 01:50:35,128 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=5 01:50:35,128 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:35,128 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 119.8 μs 01:50:35,128 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 01:50:35,128 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:35,128 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:35,128 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 01:50:35,128 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:50:35,129 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 Queuing TransactionOperation 01:50:35,129 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 01:50:35,129 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:50:35,129 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@59cbef09 01:50:35,130 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_26#-1434678118] 01:50:35,130 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_26#-1434678118] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:35,130 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_26#-1434678118, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0, version=13] 01:50:35,130 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:35,130 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_26#-1434678118)] 01:50:35,132 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 ReadData operation succeeded 01:50:35,132 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 closeTransaction called 01:50:35,132 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_26#-1434678118)] 01:50:35,132 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:50:35,132 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:35,132 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:35,132 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 01:50:35,132 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}] 01:50:35,132 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 Readying 1 components for commit 01:50:35,132 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 Readying transaction for shard cars 01:50:35,132 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 directCommit called 01:50:35,132 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 sending 1 batched modifications, ready: true 01:50:35,132 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:50:35,132 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 canCommit 01:50:35,133 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:50:35,133 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@59cbef09 01:50:35,133 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:50:35,133 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 of 1 operations, client version 13 01:50:35,133 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:50:35,133 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 ready 01:50:35,133 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}], operation=WRITE}}}}}}}]} 01:50:35,133 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:50:35,133 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 validated 01:50:35,134 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:35,134 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: success 01:50:35,134 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:50:35,134 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:50:35,134 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 candidate ready 01:50:35,134 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: canCommit - candidate rootPath: / 01:50:35,134 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: canCommit - messages: [] 01:50:35,134 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: preCommit - successfulFromPrevious: [] 01:50:35,134 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 prepared 01:50:35,134 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:35,134 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:50:35,134 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:50:35,134 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 01:50:35,134 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 01:50:35,134 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 01:50:35,134 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:35,134 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 6, leaderLastIndex: 6, leaderSnapShotIndex: 4 01:50:35,134 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 6 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:35,134 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}]]] 01:50:35,134 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 submitted to persistence 01:50:35,135 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 01:50:35,135 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}]]] 01:50:35,135 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:50:35,135 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 5, entries to be added from: 0 01:50:35,135 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 01:50:35,135 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 01:50:35,135 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 01:50:35,135 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:35,136 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 01:50:35,136 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 6, nextIndex: 7 01:50:35,136 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 5 01:50:35,136 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 6 01:50:35,136 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 5 to 6 01:50:35,136 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 6 data Simple{size=240} 01:50:35,136 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:50:35,136 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: commit - successfulFromPrevious: [] 01:50:35,136 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 changing state from COMMIT_PENDING to COMMITTED 01:50:35,136 PM [cluster-test-shard-dispatcher-28] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:50:35,136 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$k] 01:50:35,136 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 5 for term 4 01:50:35,137 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 01:50:35,137 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=6 01:50:35,137 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:35,137 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 68.93 μs 01:50:35,137 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 modifications completed with CommitTransactionReply [version=13] 01:50:35,137 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 successfully completed direct commit 01:50:35,138 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 01:50:35,138 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:35,138 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:35,138 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 01:50:35,138 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:50:35,138 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 Queuing TransactionOperation 01:50:35,138 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 6 01:50:35,138 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 5, lastIndex(): 6 01:50:35,138 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 6 01:50:35,138 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 5 to 6 01:50:35,138 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 6 data Simple{size=240} 01:50:35,138 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:50:35,139 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:35,139 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 5 for term 4 01:50:35,139 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=6 01:50:35,139 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 78.60 μs 01:50:35,142 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 01:50:35,142 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:50:35,142 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@2daa048f 01:50:35,142 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_27#-532581029] 01:50:35,142 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_27#-532581029] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:35,142 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_27#-532581029, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0, version=13] 01:50:35,143 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:35,143 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_27#-532581029)] 01:50:35,144 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 ReadData operation succeeded 01:50:35,144 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 closeTransaction called 01:50:35,144 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_27#-532581029)] 01:50:35,144 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:50:35,144 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:35,144 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:35,144 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 01:50:35,145 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}] 01:50:35,145 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 Readying 1 components for commit 01:50:35,145 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 Readying transaction for shard cars 01:50:35,145 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 directCommit called 01:50:35,145 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 sending 1 batched modifications, ready: true 01:50:35,145 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:50:35,145 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 canCommit 01:50:35,145 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:50:35,145 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@2daa048f 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 of 1 operations, client version 13 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 ready 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}], operation=WRITE}}}}}}}]} 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 validated 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: success 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 candidate ready 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: canCommit - candidate rootPath: / 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: canCommit - messages: [] 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: preCommit - successfulFromPrevious: [] 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 prepared 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: 5 01:50:35,146 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 7 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:35,147 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}]]] 01:50:35,147 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 submitted to persistence 01:50:35,147 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 01:50:35,148 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}]]] 01:50:35,148 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:50:35,148 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 01:50:35,148 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 01:50:35,148 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 01:50:35,148 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 01:50:35,148 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=7, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:35,148 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 01:50:35,149 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 7, nextIndex: 8 01:50:35,149 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 6 01:50:35,149 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 7 01:50:35,149 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 6 to 7 01:50:35,149 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 7 data Simple{size=240} 01:50:35,149 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:50:35,149 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: commit - successfulFromPrevious: [] 01:50:35,149 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 changing state from COMMIT_PENDING to COMMITTED 01:50:35,149 PM [cluster-test-shard-dispatcher-28] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:50:35,149 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$n] 01:50:35,149 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:35,149 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 81.61 μs 01:50:35,149 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 6 for term 4 01:50:35,149 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 01:50:35,149 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=7 01:50:35,150 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 modifications completed with CommitTransactionReply [version=13] 01:50:35,150 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 successfully completed direct commit 01:50:35,150 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 01:50:35,150 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:35,150 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:35,150 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 01:50:35,150 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:50:35,150 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 Queuing TransactionOperation 01:50:35,151 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 7 01:50:35,151 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 7, context.getLastApplied(): 6, lastIndex(): 7 01:50:35,151 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 7 01:50:35,151 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 6 to 7 01:50:35,151 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 7 data Simple{size=240} 01:50:35,151 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:50:35,152 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:50:35,152 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 6 for term 4 01:50:35,152 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=7 01:50:35,152 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 110.7 μs 01:50:35,152 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 01:50:35,152 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:50:35,152 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@779dcd9d 01:50:35,152 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_28#638098212] 01:50:35,152 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_28#638098212] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:35,153 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_28#638098212, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0, version=13] 01:50:35,153 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:35,153 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_28#638098212)] 01:50:35,155 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 ReadData operation succeeded 01:50:35,156 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 closeTransaction called 01:50:35,156 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_28#638098212)] 01:50:35,657 PM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-1222322845] 01:50:35,657 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: findLocalShard : cars 01:50:35,658 PM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1108744531] 01:50:35,658 PM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.TestShard$RequestFrontendMetadata to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1108744531] 01:50:35,658 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 01:50:35,659 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:35,659 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:35,659 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 01:50:35,659 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 01:50:35,659 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 Queuing TransactionOperation 01:50:35,661 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 01:50:35,661 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 01:50:35,661 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@779dcd9d 01:50:35,661 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-11_29#1455196216] 01:50:35,662 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-11_29#1455196216, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0, version=13] 01:50:35,662 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:50:35,662 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-11_29#1455196216)] 01:50:35,664 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 ReadData operation succeeded 01:50:35,665 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 closeTransaction called 01:50:35,665 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-11_29#1455196216)] 01:50:35,665 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:50:35,666 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Issuing gracefulStop to shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:35,666 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession - waiting on 1 shards 01:50:35,666 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onShutDown 01:50:35,666 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Initiating leader transfer 01:50:35,666 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=null, leaderPayloadVersion=13] 01:50:35,666 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@21cbc53a 01:50:35,666 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@21cbc53a 01:50:35,666 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Attempting to transfer leadership 01:50:35,666 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:35,666 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:35,666 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 01:50:35,668 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession] 01:50:35,668 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=null, leaderPayloadVersion=13] 01:50:35,668 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:35,668 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:35,668 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, matchIndex: 7, lastIndex: 7, isVoting: true 01:50:35,668 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Follower's log matches - sending ElectionTimeout 01:50:35,668 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 01:50:35,669 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Leader transfer complete 01:50:35,871 PM [cluster-test-shard-dispatcher-11] [WARN] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=7, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 201, lastApplied : 7, commitIndex : 7 01:50:35,901 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:50:36,79 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All shards shutdown - sending PoisonPill to self 01:50:36,79 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1108744531]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:36,79 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:36,80 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader)_slicer-id-513: Closing 01:50:36,80 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborting 0 pending queued transactions 01:50:36,80 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 01:50:36,80 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 01:50:36,80 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:50:36,81 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 01:50:36,81 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 01:50:36,81 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:36,81 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:36,88 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:36,88 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:36,88 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:36,88 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:36,90 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:36,101 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 01:50:36,101 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:50:36,101 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Closing 01:50:36,101 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborting 0 pending queued transactions 01:50:36,101 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 01:50:36,101 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:36,101 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:37,96 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-6490653483033289381] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:37,117 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-8431985092865729004] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:37,125 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:37,125 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:37,127 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:37.128] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:37,141 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:37,141 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:37,143 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:37.143] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:37,165 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:37,172 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-5380258564941347070] 01:50:37,173 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:37,181 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:37,181 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:37,183 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:37,183 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:37,183 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:37,183 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:37,185 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:37,188 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:37,194 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-2188892222249151879] 01:50:37,194 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:37,197 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:37,197 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:37,203 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:37,203 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:37,216 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:37,226 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-578960011877497046] 01:50:37,226 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:37,228 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:37,228 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:37,228 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:37,228 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:37,239 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:37,240 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:37,242 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 01:50:37,243 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:37,250 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:37,253 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:50:37,254 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#158847557] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:37,258 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 01:50:37,258 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:37,258 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 01:50:37,258 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:50:37,258 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7687e783 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@ec69ee5 01:50:37,258 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@37416d35 01:50:37,258 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:37,258 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:37,260 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1710165037260, None)) 01:50:37,260 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165037259,0,0)) 01:50:37,261 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@ec69ee5 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3f81680b 01:50:37,261 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 01:50:37,261 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1d4029e3 01:50:37,261 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:37,261 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:37,261 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:37,261 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:37,261 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:37,261 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1589397233] 01:50:37,262 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 01:50:37,262 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1589397233] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:37,262 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-1:snapshot-read#-385231691] 01:50:37,262 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-2028248792 created and ready for shard:member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:37,262 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-2028248792] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:37,262 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-1:snapshot-read#-385231691] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:37,262 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 01:50:37,262 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:37,263 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:37,263 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:37,264 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 01:50:37,264 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:50:37,264 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 01:50:37,264 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:50:37,264 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 01:50:37,264 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 01:50:37,264 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 01:50:37,264 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 01:50:37,265 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:37,268 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-442608469] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:37,268 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:50:37,269 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 01:50:37,269 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:50:37,269 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@63764cf4 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1e41254f 01:50:37,270 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@678360b8 01:50:37,270 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1710165037269, None)) 01:50:37,270 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 01:50:37,270 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:37,270 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:37,270 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:37,270 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:37,271 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165037268,0,0)) 01:50:37,271 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1e41254f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@10139c7d 01:50:37,271 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 01:50:37,271 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3f9a0804 01:50:37,272 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:37,272 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:37,273 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:37,273 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:37,273 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop#-945740983] 01:50:37,273 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 01:50:37,273 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop#-945740983] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:37,273 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-1611375311 created and ready for shard:member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:37,273 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 01:50:37,274 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-1611375311] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:37,274 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-2:snapshot-read#-1537449874] 01:50:37,274 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-2:snapshot-read#-1537449874] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:37,274 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:37,274 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:37,274 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 01:50:37,274 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 01:50:37,274 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:50:37,274 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:50:37,274 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 01:50:37,274 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:37,274 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 01:50:37,274 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 01:50:37,322 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#158847557] 01:50:37,322 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: findLocalShard : cars 01:50:37,322 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1589397233] 01:50:37,532 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:37,532 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Received ElectionTimeout - switching to Candidate 01:50:37,532 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:50:37,532 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=1, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:37,532 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 1 01:50:37,532 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:37,532 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 01:50:37,532 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:50:37,532 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:50:37,532 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Follower to Candidate 01:50:37,532 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Follower to Candidate 01:50:37,782 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 01:50:37,782 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=2, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:37,782 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 2 01:50:37,805 PM [cluster-test-shard-dispatcher-26] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:38,11 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 01:50:38,11 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=3, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,12 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 3 01:50:38,202 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:38,202 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:38,203 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:38,203 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,203 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,203 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:38,203 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,203 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,203 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,203 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,209 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:38,209 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,209 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,209 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:38,209 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,209 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,209 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,241 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:38,292 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 01:50:38,292 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,292 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 4 01:50:38,292 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,292 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,293 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:38,293 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 01:50:38,293 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:38,293 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:38,293 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=null 01:50:38,293 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:38,293 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,293 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:38,294 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1589397233] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:38,294 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:38,294 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:38,294 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:50:38,294 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:38,294 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:38,295 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:38,295 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 01:50:38,295 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: became leader with frontend state for [] 01:50:38,295 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Candidate to Leader 01:50:38,295 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 01:50:38,295 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:38,295 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:38,295 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Candidate to Leader 01:50:38,295 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 01:50:38,295 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1589397233] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:38,296 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 01:50:38,296 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: getLeaderAddress leaderId = member-1-shard-cars-testRaftCallbackDuringLeadershipDrop peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,296 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done false 01:50:38,296 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 01:50:38,296 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:38,296 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:38,296 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 01:50:38,296 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 01:50:38,296 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:38,327 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testRaftCallbackDuringLeadershipDrop commit rateLimit adjusted to 162.12298098119095 01:50:38,327 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:50:38,327 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#158847557] 01:50:38,327 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 Queuing TransactionOperation 01:50:38,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:38,327 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 Readying 1 components for commit 01:50:38,327 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 Readying transaction for shard cars 01:50:38,327 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 Queuing TransactionOperation 01:50:38,327 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 canCommit 01:50:38,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1589397233, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@718474b7] 01:50:38,327 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1589397233, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@718474b7] 01:50:38,327 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1589397233)] for shard cars 01:50:38,327 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 01:50:38,327 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@764bc89c 01:50:38,328 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 = [ 1 ] 01:50:38,328 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1589397233)] 01:50:38,328 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: handleReadyLocalTransaction for member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:50:38,328 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, disabling transaction tracking 01:50:38,328 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 01:50:38,328 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}]]] 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying local modifications for Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Validating transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 validated 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: canCommit for member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: success 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Committing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Preparing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 candidate ready 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: canCommit - messages: [] 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 prepared 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Finishing commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Starting commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 173 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:50:38,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:38,330 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:38,330 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:38,330 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,330 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 submitted to persistence 01:50:38,330 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 01:50:38,331 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:50:38,331 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1589397233] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:38,331 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1589397233] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:38,331 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}]]] 01:50:38,331 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 1 01:50:38,331 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:38,331 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 01:50:38,331 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 01:50:38,331 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 1 01:50:38,331 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:38,332 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 01:50:38,332 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done true 01:50:38,336 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 0, nextIndex: 1 01:50:38,336 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:38,336 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Setting last applied to 0 01:50:38,336 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 01:50:38,336 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 01:50:38,336 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:50:38,336 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} does not match any client, pre-disabling client. 01:50:38,336 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, nanosAgo=7944543} is already disabled 01:50:38,336 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:38,336 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,336 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 01:50:38,337 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=0 01:50:38,340 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 01:50:38,340 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 1 01:50:38,340 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:38,340 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=173} 01:50:38,340 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:50:38,340 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 2 01:50:38,340 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 0 01:50:38,340 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:38,340 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:50:38,340 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 0 01:50:38,340 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 01:50:38,340 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 01:50:38,340 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:50:38,340 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} does not match any client, pre-disabling client. 01:50:38,341 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:50:38,341 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=0 01:50:38,341 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 1, nextIndex: 2 01:50:38,341 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:38,341 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Setting last applied to 1 01:50:38,341 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 01:50:38,341 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data Simple{size=173} 01:50:38,341 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Resuming commit of transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:50:38,341 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:50:38,341 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:50:38,341 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testRaftCallbackDuringLeadershipDrop_member-1-shard-cars-testRaftCallbackDuringLeadershipDrop%231589397233$a] 01:50:38,341 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 committed successfully 01:50:38,341 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 successfully completed direct commit 01:50:38,341 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-341597991] with name member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 01:50:38,341 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-341597991] 01:50:38,341 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:50:38,341 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:38,342 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 01:50:38,342 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-341597991] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:38,342 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 79.98 μs 01:50:38,342 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:38,343 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:38,344 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 01:50:38,346 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 1 01:50:38,346 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:38,346 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 1 01:50:38,346 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 01:50:38,346 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data Simple{size=173} 01:50:38,346 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:50:38,346 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#823761193] with name member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 01:50:38,346 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 01:50:38,346 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#823761193] 01:50:38,346 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:50:38,347 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 70.70 μs 01:50:38,347 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#823761193] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:38,350 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1701415327] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:38,350 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:50:38,353 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 01:50:38,353 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:50:38,353 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7d5e0546 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2ec55e5c 01:50:38,353 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6bc7c5e0 01:50:38,353 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 01:50:38,353 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,354 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1710165038353, None)) 01:50:38,354 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:38,354 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,354 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:38,354 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165038352,0,0)) 01:50:38,354 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,354 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:38,354 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,354 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2ec55e5c to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@710d3173 01:50:38,354 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 01:50:38,354 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1701415327] 01:50:38,355 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6aba7269 01:50:38,355 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:38,355 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,355 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,355 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,355 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#1367739919] 01:50:38,355 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,355 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,355 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#1367739919] 01:50:38,355 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,355 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,355 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#1367739919] 01:50:38,355 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-442608469] 01:50:38,356 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:50:38,356 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 01:50:38,356 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#1367739919] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:38,356 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1589397233)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:38,356 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@764bc89c 01:50:38,356 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:38,356 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#1287967597 created and ready for shard:member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,356 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#1287967597] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:38,356 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-3:snapshot-read#2071964221] 01:50:38,356 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-3:snapshot-read#2071964221] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:38,357 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 01:50:38,358 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:38,358 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:38,358 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 01:50:38,358 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 01:50:38,358 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:50:38,358 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:50:38,358 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 01:50:38,358 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,358 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,358 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 01:50:38,358 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 01:50:38,361 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#158847557] 01:50:38,361 PM [pool-34-thread-1] [DEBUG] TransactionProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 Readying 1 components for commit 01:50:38,361 PM [pool-34-thread-1] [DEBUG] TransactionProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 Readying transaction for shard cars 01:50:38,361 PM [pool-34-thread-1] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 = [ 1 ] 01:50:38,361 PM [pool-34-thread-1] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1589397233)] 01:50:38,362 PM [pool-34-thread-1] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 canCommit 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: handleReadyLocalTransaction for member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying local modifications for Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Validating transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 validated 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: canCommit for member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: success 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Committing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Preparing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 candidate ready 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: canCommit - candidate rootPath: / 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: canCommit - messages: [] 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 prepared 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Finishing commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Starting commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 71 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}]]] 01:50:38,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 submitted to persistence 01:50:38,364 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 01:50:38,462 PM [awaitility-thread] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1589397233] 01:50:38,463 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1589397233] 01:50:38,464 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:38,464 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Cluster state: [] 01:50:38,464 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:38,464 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Term 5 in "RequestVote [term=5, candidateId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1]" message is greater than leader's term 4 - switching to Follower 01:50:38,464 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=5, votedFor=null 01:50:38,464 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader) :- Switching from behavior Leader to Follower, election term: 5 01:50:38,464 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader)_slicer-id-518: Closing 01:50:38,464 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:50:38,464 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:50:38,464 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: removing frontend state for [member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop] 01:50:38,464 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 01:50:38,465 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Leader to Follower 01:50:38,465 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=-1] 01:50:38,465 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:38,465 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:38,465 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Leader to Follower 01:50:38,477 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 closeTransaction called 01:50:38,477 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_22#-962814835)] 01:50:38,477 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 closeTransaction called 01:50:38,477 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_23#-1386525328)] 01:50:38,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:38,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Received ElectionTimeout - switching to Candidate 01:50:38,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:50:38,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,772 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 6 01:50:38,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] to peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] to peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,772 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower) :- Switching from behavior Follower to Candidate, election term: 6 01:50:38,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 01:50:38,773 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:50:38,773 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:50:38,773 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Follower to Candidate 01:50:38,773 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Follower to Candidate 01:50:38,774 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:38,774 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 01:50:38,774 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:38,774 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:38,774 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 6 in "RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 4 - updating term 01:50:38,774 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 01:50:38,774 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=null 01:50:38,774 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 01:50:38,774 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 6 in "RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 0 - updating term 01:50:38,774 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=null 01:50:38,774 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 1, lastTerm: 4 01:50:38,774 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:38,774 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,774 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,774 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 01:50:38,774 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 01:50:38,776 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=true], my term is 6 01:50:38,776 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=true], current voteCount: 1 01:50:38,776 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): LastApplied index 1 is behind last index 2 - switching to PreLeader 01:50:38,782 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:50:38,782 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:38,783 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:38,783 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): The followers log is empty and the senders prevLogIndex is 0 01:50:38,783 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:38,783 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 01:50:38,783 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: getLeaderAddress leaderId = member-1-shard-cars-testRaftCallbackDuringLeadershipDrop peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,784 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 01:50:38,784 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:38,784 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:38,784 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 01:50:38,784 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done false 01:50:38,785 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 6 01:50:38,785 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 01:50:38,785 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: became leader with frontend state for [member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop] 01:50:38,785 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 01:50:38,785 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:38,785 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Candidate to PreLeader 01:50:38,785 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:38,785 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Candidate to PreLeader 01:50:38,785 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] 01:50:38,785 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] 01:50:38,785 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Replicate message: identifier: null, logIndex: 3, payload: class org.opendaylight.controller.cluster.raft.persisted.NoopPayload, isSendImmediate: true 01:50:38,785 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:50:38,785 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:50:38,785 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 01:50:38,785 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 01:50:38,786 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 01:50:38,786 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:50:38,786 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,786 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e]]] 01:50:38,786 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 1, nextIndex: 2 01:50:38,786 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:50:38,786 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,786 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e]]] 01:50:38,786 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] 01:50:38,787 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): The followers log is empty and the senders prevLogIndex is 0 01:50:38,787 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:38,787 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done true 01:50:38,791 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 01:50:38,791 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 01:50:38,792 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e]]] 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 4 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=173} 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=71} 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 1 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 3 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 0 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} does not match any client, pre-disabling client. 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 1 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data Simple{size=173} 01:50:38,793 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:50:38,793 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 3, nextIndex: 4 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 4 does not match the current term 6 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 01:50:38,794 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#1621895733] with name member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Setting last applied to 2 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0, size=71} 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Resuming commit of transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: commit - successfulFromPrevious: [] 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testRaftCallbackDuringLeadershipDrop_member-1-shard-cars-testRaftCallbackDuringLeadershipDrop%231589397233$b] 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Setting last applied to 3 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 1 for term 6 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 1 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Received ApplyState [identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}]] - lastApplied: 3, lastIndex: 3 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 1 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 1 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e]]] 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 6 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader)_slicer-id-521: Closing 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 01:50:38,794 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 01:50:38,795 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#1621895733] 01:50:38,795 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:50:38,795 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#1621895733] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:38,795 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 183.0 μs 01:50:38,795 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 committed successfully 01:50:38,795 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:50:38,795 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 successfully completed direct commit 01:50:38,795 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from PreLeader to Leader 01:50:38,795 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 12.02 μs 01:50:38,796 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from PreLeader to Leader 01:50:38,796 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 01:50:38,796 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e]]] 01:50:38,796 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 2 01:50:38,796 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:50:38,796 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=71} 01:50:38,796 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 01:50:38,796 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e 01:50:38,796 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] 01:50:38,796 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 01:50:38,796 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:38,796 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,797 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 3, nextIndex: 4 01:50:38,797 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 2 for term 6 01:50:38,797 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:50:38,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 01:50:38,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] 01:50:38,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e]]] 01:50:38,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 2 01:50:38,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] is present: existingEntryTerm: 4 01:50:38,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] is present: existingEntryTerm: 6 01:50:38,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 3, entries to be added from: 2 01:50:38,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 01:50:38,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 3 01:50:38,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:38,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 01:50:38,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 2 01:50:38,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 01:50:38,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{size=71} 01:50:38,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:50:38,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 3 01:50:38,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 01:50:38,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e 01:50:38,797 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:50:38,797 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 01:50:38,797 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 23.00 μs 01:50:38,797 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 01:50:38,797 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 01:50:38,797 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 01:50:38,798 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] 01:50:38,798 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 01:50:38,798 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 3 01:50:38,798 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 01:50:38,798 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 2 01:50:38,798 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 01:50:38,798 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{size=71} 01:50:38,798 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:50:38,798 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:50:38,798 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 3 01:50:38,798 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 01:50:38,798 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e 01:50:38,798 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 1 for term 6 01:50:38,798 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 22.98 μs 01:50:38,798 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 01:50:38,869 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:50:38,870 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,870 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 01:50:38,870 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 01:50:38,870 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,870 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 01:50:38,870 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 01:50:38,870 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 01:50:38,871 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:50:38,871 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:50:38,871 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 01:50:38,871 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:50:38,871 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1701415327] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1701415327] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:38,872 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:50:38,872 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:38,872 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 01:50:38,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 01:50:38,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Initiating leader transfer 01:50:38,872 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=13] 01:50:38,872 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:38,872 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:38,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@11329bf 01:50:38,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@11329bf 01:50:38,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Attempting to transfer leadership 01:50:38,873 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:50:38,873 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:50:38,875 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:50:38,875 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 2 for term 6 01:50:38,876 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=13] 01:50:38,876 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:38,876 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:38,878 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, matchIndex: 3, lastIndex: 3, isVoting: true 01:50:38,878 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Follower's log matches - sending ElectionTimeout 01:50:38,878 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:50:38,878 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:50:38,878 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Leader transfer complete 01:50:39,222 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:50:39,291 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 01:50:39,291 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1589397233]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:39,291 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:39,291 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader)_slicer-id-522: Closing 01:50:39,291 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 01:50:39,291 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:50:39,292 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:50:39,292 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:50:39,292 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 01:50:39,292 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:50:39,292 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:50:39,292 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#158847557] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#158847557] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:39,292 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#158847557] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#158847557] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:39,292 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#158847557] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#158847557] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:39,292 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#-1033237032] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1033237032] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:39,292 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#-1033237032] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1033237032] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:39,293 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:39,293 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:39,296 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:39,296 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:39,296 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:39,296 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:39,298 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:39,314 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:50:39,315 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:50:39,315 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 01:50:39,315 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 01:50:39,315 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:50:39,316 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:39,316 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:40,311 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-5380258564941347070] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:40,331 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-2188892222249151879] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:40,337 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:40,337 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:40,339 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:40.340] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:40,357 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:40,357 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:40,359 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:40.361] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:40,391 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:50:40,396 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [3250364858043607067] 01:50:40,397 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:40,399 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:40,399 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:40,404 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:40,404 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:40,408 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:40,408 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:40,408 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:40,411 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:40,415 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4165655297989269610] 01:50:40,415 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:40,417 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:40,417 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:40,428 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:40,428 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:40,433 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:40,440 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2182168754401356040] 01:50:40,441 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:40,441 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:40,441 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:40,452 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:40,452 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:40,453 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:40,454 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:40,455 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithSingleShard 01:50:40,466 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1702509462] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:40,467 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithSingleShard 01:50:40,468 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: no initial generation override, starting from 0 01:50:40,468 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:50:40,468 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1333203a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7080d56b 01:50:40,468 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@74bf4fa3 01:50:40,468 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithSingleShard 01:50:40,468 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Creating local shard: member-1-shard-cars-testTransactionChainWithSingleShard 01:50:40,469 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:40,469 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:50:40,469 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionChainWithSingleShard, 0, 1710165040469, None)) 01:50:40,471 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165040468,0,0)) 01:50:40,471 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7080d56b to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@22d3412e 01:50:40,471 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@134bfbfa 01:50:40,471 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} started 01:50:40,471 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:40,473 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:40,476 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithSingleShard 01:50:40,476 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:40,476 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionChainWithSingleShard 01:50:40,477 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:50:40,477 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:50:40,477 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: updated peer member-1-shard-cars-testTransactionChainWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard on shard member-1-shard-cars-testTransactionChainWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1624534874] 01:50:40,477 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionChainWithSingleShard, persistent : true 01:50:40,477 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionChainWithSingleShard with journal batch size 1 01:50:40,480 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithSingleShard 01:50:40,480 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1898740210] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:40,485 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithSingleShard 01:50:40,485 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: no initial generation override, starting from 0 01:50:40,485 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Creating local shard: member-2-shard-cars-testTransactionChainWithSingleShard 01:50:40,485 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:50:40,485 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6f1e8172 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2b48208f 01:50:40,486 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@764d18bf 01:50:40,486 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionChainWithSingleShard, 0, 1710165040486, None)) 01:50:40,486 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165040485,0,0)) 01:50:40,486 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2666f6a7 01:50:40,487 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2b48208f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4a3c627d 01:50:40,487 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} started 01:50:40,487 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:40,487 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionChainWithSingleShard 01:50:40,487 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionChainWithSingleShard, persistent : true 01:50:40,488 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1624534874] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:40,488 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-notifier#-2120347626 created and ready for shard:member-1-shard-cars-testTransactionChainWithSingleShard 01:50:40,488 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-member-1:snapshot-read#932176943] 01:50:40,488 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-notifier#-2120347626] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:40,488 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:40,488 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionChainWithSingleShard with journal batch size 1 01:50:40,488 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-member-1:snapshot-read#932176943] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:40,488 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:40,488 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:40,488 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard#1545379721] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:40,489 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/shard-member-2:snapshot-read#939973329] 01:50:40,489 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:40,489 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithSingleShard is no longer the leader 01:50:40,489 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Initializing shard [cars] 01:50:40,489 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionChainWithSingleShard is initialized 01:50:40,489 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from null to Follower 01:50:40,489 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard 01:50:40,489 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-notifier#-480585713] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:40,489 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from null to Follower 01:50:40,493 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:40,493 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:40,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Initializing shard [cars] 01:50:40,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionChainWithSingleShard is initialized 01:50:40,493 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:40,493 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionChainWithSingleShard is no longer the leader 01:50:40,496 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:40,496 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:40,497 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:50:40,497 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:50:40,497 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionChainWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:50:40,498 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-notifier#-480585713 created and ready for shard:member-2-shard-cars-testTransactionChainWithSingleShard 01:50:40,498 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithSingleShard , received role change from null to Follower 01:50:40,498 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard 01:50:40,498 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-2-shard-cars-testTransactionChainWithSingleShard from null to Follower 01:50:40,503 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:40,504 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:40,506 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/shard-member-2:snapshot-read#939973329] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:40,537 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1702509462] 01:50:40,537 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: findLocalShard : cars 01:50:40,537 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1624534874] 01:50:40,797 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:40,797 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 01:50:40,797 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionChainWithSingleShard] 01:50:40,797 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 01:50:40,797 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 1 01:50:40,797 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:40,797 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Closing 01:50:40,797 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:40,797 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithSingleShard is no longer the leader 01:50:40,798 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from Follower to Candidate 01:50:40,798 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from Follower to Candidate 01:50:41,77 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 01:50:41,77 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 01:50:41,77 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 2 01:50:41,100 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:41,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 01:50:41,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 01:50:41,327 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 3 01:50:41,418 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:41,418 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:41,419 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:41,420 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard 01:50:41,420 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:50:41,420 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:41,420 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:50:41,420 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:50:41,420 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionChainWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:50:41,428 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:41,428 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard 01:50:41,428 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:50:41,428 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:41,428 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:50:41,428 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:50:41,468 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:41,547 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 01:50:41,547 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 01:50:41,547 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 4 01:50:41,547 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionChainWithSingleShard 01:50:41,549 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:41,549 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Cluster state: [] 01:50:41,549 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:41,549 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:41,549 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=null 01:50:41,549 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:41,549 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 01:50:41,549 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:41,551 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:41,551 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:41,551 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionChainWithSingleShard] 01:50:41,551 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:41,551 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:41,551 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:50:41,551 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: became leader with frontend state for [] 01:50:41,551 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from Candidate to Leader 01:50:41,551 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderPayloadVersion=13] 01:50:41,551 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:41,551 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:41,551 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from Candidate to Leader 01:50:41,551 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: All Shards are ready - data store testTransactionChainWithSingleShard is ready 01:50:41,553 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithSingleShard: Closing 01:50:41,554 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testTransactionChainWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:50:41,554 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard Received follower initial sync status for member-2-shard-cars-testTransactionChainWithSingleShard status sync done false 01:50:41,554 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderPayloadVersion=13] 01:50:41,554 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:41,554 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:41,554 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: All Shards are ready - data store testTransactionChainWithSingleShard is ready 01:50:41,554 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: -1, nextIndex: 0 01:50:41,554 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:41,595 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionChainWithSingleShard commit rateLimit adjusted to 162.12298098119095 01:50:41,595 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:50:41,595 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1898740210] 01:50:41,595 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:50:41,595 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Readying 1 components for commit 01:50:41,595 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 01:50:41,595 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:41,595 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:50:41,596 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard, primaryVersion=13] 01:50:41,596 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 01:50:41,596 PM [main] [DEBUG] TransactionChainProxy - Tx: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 - waiting for ready futures with pending Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:50:41,596 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard, primaryVersion=13] 01:50:41,596 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 Queuing TransactionOperation 01:50:41,596 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] for shard cars 01:50:41,596 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] found - creating WRITE_ONLY transaction context 01:50:41,596 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:41,596 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 directCommit called 01:50:41,596 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 sending 1 batched modifications, ready: true 01:50:41,596 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] 01:50:41,598 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, disabling transaction tracking 01:50:41,598 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 01:50:41,598 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 01:50:41,598 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:41,598 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:41,598 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 01:50:41,598 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}]]] 01:50:41,598 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 01:50:41,598 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 01:50:41,598 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:50:41,598 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} 01:50:41,598 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@3935d4c1 01:50:41,598 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying 1 batched modifications for Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Readying Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 of 1 operations, client version 13 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 ready 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE}}}]} 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Validating transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 validated 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: canCommit for member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: success 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Committing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Preparing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 candidate ready 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: canCommit - candidate rootPath: / 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: canCommit - messages: [] 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: preCommit - successfulFromPrevious: [] 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 prepared 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Finishing commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Starting commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 173 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}] 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}] 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 01:50:41,599 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 submitted to persistence 01:50:41,600 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}]]] 01:50:41,600 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 01:50:41,600 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 1 01:50:41,600 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:41,600 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62} 01:50:41,600 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 01:50:41,600 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 01:50:41,600 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}] 01:50:41,601 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 1 01:50:41,601 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:41,601 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 01:50:41,601 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard Received follower initial sync status for member-2-shard-cars-testTransactionChainWithSingleShard status sync done true 01:50:41,601 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 0, nextIndex: 1 01:50:41,601 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:41,601 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 0 01:50:41,601 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from -1 to 0 01:50:41,602 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62} 01:50:41,602 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:50:41,602 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionChainWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} does not match any client, pre-disabling client. 01:50:41,602 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, nanosAgo=4002942} is already disabled 01:50:41,602 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:50:41,602 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 01:50:41,602 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}]]] 01:50:41,602 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=0 01:50:41,603 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}]]] 01:50:41,603 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 2 01:50:41,603 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:41,603 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64} 01:50:41,603 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 01:50:41,603 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log Simple{size=173} 01:50:41,603 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}] 01:50:41,603 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 3 01:50:41,603 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 0 01:50:41,603 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:41,603 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 2 01:50:41,603 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 0 01:50:41,603 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from -1 to 0 01:50:41,603 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62} 01:50:41,603 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:50:41,603 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionChainWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} does not match any client, pre-disabling client. 01:50:41,603 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 01:50:41,603 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}] 01:50:41,604 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=0 01:50:41,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 2, nextIndex: 3 01:50:41,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 01:50:41,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 1 01:50:41,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 0 to 1 01:50:41,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64} 01:50:41,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} has no callback 01:50:41,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 2 01:50:41,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 1 to 2 01:50:41,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 2 data Simple{size=173} 01:50:41,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Resuming commit of transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:50:41,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: commit - successfulFromPrevious: [] 01:50:41,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:50:41,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 01:50:41,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithSingleShard_member-1-shard-cars-testTransactionChainWithSingleShard$a] 01:50:41,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionChainWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#796565604] with name member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher 01:50:41,604 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#796565604] 01:50:41,604 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 1 for term 4 01:50:41,605 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:41,605 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 01:50:41,605 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=2 01:50:41,605 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#796565604] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:41,605 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 107.3 μs 01:50:41,605 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionChainProxy - Tx: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 - previous Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 readied - proceeding to FindPrimaryShard 01:50:41,605 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] for shard cars 01:50:41,605 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] found - trying create transaction 01:50:41,605 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] 01:50:41,605 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 2 01:50:41,605 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 modifications completed with CommitTransactionReply [version=13] 01:50:41,605 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 01:50:41,605 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 1 01:50:41,605 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 0 to 1 01:50:41,605 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64} 01:50:41,606 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 2 01:50:41,606 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 1 to 2 01:50:41,606 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 2 data Simple{size=173} 01:50:41,606 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithSingleShard: Applying foreign transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:50:41,606 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionChainWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-139283382] with name member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher 01:50:41,606 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-139283382] 01:50:41,606 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 1 for term 4 01:50:41,606 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 01:50:41,606 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=2 01:50:41,606 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 59.21 μs 01:50:41,606 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-139283382] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:41,609 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Creating transaction : member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 01:50:41,609 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} 01:50:41,609 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@dbc3aa5 01:50:41,609 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_30#1750808237] 01:50:41,610 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_30#1750808237] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:41,610 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_30#1750808237, transactionId=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0, version=13] 01:50:41,610 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:41,610 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_30#1750808237)] 01:50:41,612 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 ReadData operation succeeded 01:50:41,612 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:50:41,612 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:41,612 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:41,612 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] found - trying create transaction 01:50:41,612 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] 01:50:41,612 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Queuing TransactionOperation 01:50:41,612 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Creating transaction : member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:50:41,612 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} 01:50:41,612 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@dbc3aa5 01:50:41,613 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#714534863] 01:50:41,613 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#714534863] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:41,613 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#714534863, transactionId=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, version=13] 01:50:41,613 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:41,613 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#714534863)] 01:50:41,615 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 ReadData operation succeeded 01:50:41,615 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:50:41,615 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 01:50:41,615 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeRead DataExists called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 01:50:41,615 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 sending 2 batched modifications, ready: false 01:50:41,615 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#714534863)] 01:50:41,616 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.DataExists to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#714534863)] 01:50:41,618 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 modifications completed with BatchedModificationsReply [numBatched=2] 01:50:41,618 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 DataExists operation succeeded 01:50:41,618 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:50:41,618 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#714534863)] 01:50:41,619 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 ReadData operation succeeded 01:50:41,619 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 01:50:41,619 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeDelete called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 01:50:41,619 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 canCommit 01:50:41,620 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 successfully completed direct commit 01:50:41,620 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Readying 1 components for commit 01:50:41,620 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Readying transaction for shard cars 01:50:41,620 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 directCommit called 01:50:41,620 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 sending 2 batched modifications, ready: true 01:50:41,620 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#714534863)] 01:50:41,620 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 canCommit 01:50:41,621 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:50:41,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: handleForwardedReadyTransaction for member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:50:41,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, client version 13 01:50:41,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:50:41,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 ready 01:50:41,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=MERGE, modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}], operation=MERGE}}}}}}}]} 01:50:41,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Validating transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:50:41,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 validated 01:50:41,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:41,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: canCommit for member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: success 01:50:41,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Committing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:50:41,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Preparing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:50:41,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 candidate ready 01:50:41,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: canCommit - candidate rootPath: / 01:50:41,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: canCommit - messages: [] 01:50:41,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: preCommit - successfulFromPrevious: [] 01:50:41,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 prepared 01:50:41,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:41,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Finishing commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:50:41,621 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Starting commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:50:41,622 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 229 01:50:41,622 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}] 01:50:41,622 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}] 01:50:41,622 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:41,622 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 01:50:41,622 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 01:50:41,622 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}]]] 01:50:41,622 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 submitted to persistence 01:50:41,622 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}] 01:50:41,623 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}]]] 01:50:41,623 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 1 01:50:41,623 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:50:41,623 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log Simple{size=229} 01:50:41,623 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}] 01:50:41,623 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 2 01:50:41,623 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:41,624 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}] 01:50:41,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 3, nextIndex: 4 01:50:41,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 01:50:41,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 3 01:50:41,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 2 to 3 01:50:41,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 3 data Simple{size=229} 01:50:41,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Resuming commit of transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:50:41,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: commit - successfulFromPrevious: [] 01:50:41,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 01:50:41,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=MERGE, modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}], operation=MERGE, modificationType=WRITE}}}}}}}]} 01:50:41,624 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithSingleShard_member-1-shard-cars-testTransactionChainWithSingleShard_shard-cars-member-2%3Adatastore-testTransactionChainWithSingleShard%400%3A1-2_31%23714534863$i] 01:50:41,624 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 01:50:41,625 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 2 for term 4 01:50:41,625 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:50:41,625 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=3 01:50:41,625 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 88.11 μs 01:50:41,627 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 3 01:50:41,627 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 successfully completed direct commit 01:50:41,627 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 01:50:41,627 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 3 01:50:41,627 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 2 to 3 01:50:41,627 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 3 data Simple{size=229} 01:50:41,627 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 modifications completed with CommitTransactionReply [version=13] 01:50:41,627 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 01:50:41,627 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithSingleShard: Applying foreign transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:50:41,627 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:41,627 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:41,627 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] found - trying create transaction 01:50:41,627 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] 01:50:41,627 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 01:50:41,628 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 2 for term 4 01:50:41,628 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=3 01:50:41,628 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 01:50:41,628 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 205.5 μs 01:50:41,632 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Creating transaction : member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 01:50:41,632 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} 01:50:41,632 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_32#1217732533] 01:50:41,632 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_32#1217732533] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:41,633 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_32#1217732533, transactionId=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0, version=13] 01:50:41,633 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:50:41,633 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_32#1217732533)] 01:50:41,635 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 ReadData operation succeeded 01:50:41,635 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:50:41,635 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testTransactionChainWithSingleShard 01:50:41,635 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithSingleShard - waiting on 1 shards 01:50:41,635 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onShutDown 01:50:41,635 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Initiating leader transfer 01:50:41,635 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:50:41,635 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1eb40619 01:50:41,635 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:41,635 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:41,635 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1eb40619 01:50:41,635 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Attempting to transfer leadership 01:50:41,635 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:50:41,636 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard] 01:50:41,636 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:50:41,636 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:41,636 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:41,637 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionChainWithSingleShard, matchIndex: 3, lastIndex: 3, isVoting: true 01:50:41,637 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 01:50:41,637 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:50:41,637 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Leader transfer complete 01:50:42,48 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: All shards shutdown - sending PoisonPill to self 01:50:42,48 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] ShardManager - shard-manager-testTransactionChainWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#1624534874]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:42,48 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionChainWithSingleShard 01:50:42,48 PM [cluster-test-shard-dispatcher-18] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard (Leader)_slicer-id-527: Closing 01:50:42,48 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Aborting 0 pending queued transactions 01:50:42,48 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithSingleShard 01:50:42,49 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 01:50:42,49 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:50:42,49 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 01:50:42,49 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: shutting down 01:50:42,49 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 01:50:42,50 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1702509462] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1702509462] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:42,50 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1702509462] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1702509462] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:42,50 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:42,51 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:42,59 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:42,59 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:42,59 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:42,59 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:42,68 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithSingleShard: Closing 01:50:42,69 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionChainWithSingleShard 01:50:42,69 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Closing 01:50:42,69 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Aborting 0 pending queued transactions 01:50:42,69 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithSingleShard 01:50:42,70 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:42,70 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:43,68 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [3250364858043607067] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:43,82 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:43,82 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:43,83 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:43.085] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:43,87 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4165655297989269610] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:43,92 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:43,92 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:43,94 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:43.095] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:43,116 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:43,121 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-8928914285542654510] 01:50:43,121 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:43,123 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:43,123 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:43,130 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:43,130 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:43,132 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:43,132 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:43,132 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:43,134 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:43,138 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4770318112387024822] 01:50:43,139 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:43,140 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:43,140 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:43,150 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:43,150 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:43,157 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:43,164 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-5729994039522669264] 01:50:43,166 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:43,166 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:43,166 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:43,166 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:43,166 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:43,177 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:43,177 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:43,192 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:43,193 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:43,193 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:43,196 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:43,198 PM [cluster-test-akka.actor.default-dispatcher-9] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 01:50:43,213 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:43,213 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:44,236 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:44.237] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:44,262 PM [cluster-test-akka.actor.default-dispatcher-9] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:44,262 PM [cluster-test-akka.actor.default-dispatcher-9] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:44,263 PM [cluster-test-akka.actor.default-dispatcher-9] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:44.264] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:44,291 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:44,297 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [1030504184704156361] 01:50:44,297 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:44,298 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:44,298 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:44,306 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:44,306 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:44,311 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:44,311 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:44,311 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:44,316 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:44,323 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-8171308151512354190] 01:50:44,323 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:44,324 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:44,324 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:44,329 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:44,330 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:44,338 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:44,345 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-5762576873677448900] 01:50:44,345 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:44,347 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:44,347 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:44,357 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:44,357 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:44,357 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:44,359 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:44,363 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testForwardedReadyTransactionForwardedToLeader 01:50:44,380 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-944224862] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:44,385 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:44,385 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 01:50:44,390 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:44,391 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:44,391 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testForwardedReadyTransactionForwardedToLeader 01:50:44,391 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: no initial generation override, starting from 0 01:50:44,391 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 01:50:44,391 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Creating local shard: member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:44,391 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3e754935 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@68235990 01:50:44,392 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:44,392 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:44,399 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:44,400 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@20754b37 01:50:44,400 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, 0, 1710165044392, None)) 01:50:44,402 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165044391,0,0)) 01:50:44,403 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@d114016 01:50:44,403 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@68235990 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@202b2726 01:50:44,403 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} started 01:50:44,403 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:44,406 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:44,408 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testForwardedReadyTransactionForwardedToLeader 01:50:44,408 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:44,408 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:44,409 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:44,409 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:44,409 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: updated peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-926884118] 01:50:44,409 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, persistent : true 01:50:44,409 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-926884118] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:44,409 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-1433704065 created and ready for shard:member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:44,409 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-1433704065] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:44,409 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-1:snapshot-read#-1343359317] 01:50:44,409 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with journal batch size 1 01:50:44,410 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-1:snapshot-read#-1343359317] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:44,410 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:44,410 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:44,411 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:44,411 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 01:50:44,411 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Initializing shard [cars] 01:50:44,411 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is initialized 01:50:44,411 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from null to Follower 01:50:44,411 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader 01:50:44,412 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-2095143459] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:44,414 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: no initial generation override, starting from 0 01:50:44,414 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 01:50:44,414 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@31940d8f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@11e32231 01:50:44,414 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@725f6fb7 01:50:44,415 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, 0, 1710165044414, None)) 01:50:44,415 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165044413,0,0)) 01:50:44,415 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@11e32231 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4c605ae7 01:50:44,415 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} started 01:50:44,419 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from null to Follower 01:50:44,420 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 01:50:44,424 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testForwardedReadyTransactionForwardedToLeader 01:50:44,424 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Creating local shard: member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:44,424 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@382c7814 01:50:44,424 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:44,424 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:44,425 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:44,425 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:44,425 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:44,425 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, persistent : true 01:50:44,426 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with journal batch size 1 01:50:44,426 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1168676051] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:44,426 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-2:snapshot-read#1016227766] 01:50:44,426 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-339325464 created and ready for shard:member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:44,426 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-2:snapshot-read#1016227766] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:44,427 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:44,427 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:44,427 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:44,427 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 01:50:44,427 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from null to Follower 01:50:44,427 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Initializing shard [cars] 01:50:44,427 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader is initialized 01:50:44,427 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:44,427 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader 01:50:44,427 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-339325464] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:44,427 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader from null to Follower 01:50:44,465 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-944224862] 01:50:44,466 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 01:50:44,466 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-926884118] 01:50:44,665 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:44,665 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Received ElectionTimeout - switching to Candidate 01:50:44,665 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] 01:50:44,665 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:44,665 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 1 01:50:44,665 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:44,665 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Closing 01:50:44,665 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:44,665 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 01:50:44,665 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from Follower to Candidate 01:50:44,666 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from Follower to Candidate 01:50:44,951 PM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:44,985 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:50:44,985 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:44,985 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 2 01:50:45,255 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:50:45,255 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,255 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 3 01:50:45,327 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:45,327 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:45,329 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:45,329 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,329 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,329 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:45,329 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,329 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,329 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,358 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:45,358 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,358 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,358 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:45,358 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,358 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,377 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:45,555 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:50:45,555 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,555 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 4 01:50:45,555 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,557 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:45,557 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Cluster state: [] 01:50:45,557 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:45,557 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:45,557 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=null 01:50:45,557 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:45,557 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,557 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:45,559 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:45,559 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:45,559 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] 01:50:45,559 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:45,559 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:45,559 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:50:45,560 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: became leader with frontend state for [] 01:50:45,560 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from Candidate to Leader 01:50:45,560 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderPayloadVersion=13] 01:50:45,560 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:45,560 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:45,560 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from Candidate to Leader 01:50:45,560 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All Shards are ready - data store testForwardedReadyTransactionForwardedToLeader is ready 01:50:45,561 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Closing 01:50:45,561 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,561 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader status sync done false 01:50:45,561 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderPayloadVersion=13] 01:50:45,561 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:45,561 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:45,561 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All Shards are ready - data store testForwardedReadyTransactionForwardedToLeader is ready 01:50:45,562 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: -1, nextIndex: 0 01:50:45,562 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:45,634 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-2095143459] 01:50:45,635 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 01:50:45,635 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1168676051] 01:50:45,635 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,635 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-2095143459] 01:50:45,635 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 01:50:45,635 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1168676051] 01:50:45,648 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: handleForwardedReadyTransaction for member-1-ShardTransactionTest-fe-0-txn-227-0 01:50:45,648 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,648 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader)] 01:50:45,653 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, disabling transaction tracking 01:50:45,653 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:50:45,653 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:50:45,653 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:45,653 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:45,653 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,653 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 01:50:45,654 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying 3 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-227-0 01:50:45,654 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-227-0 of 3 operations, client version 13 01:50:45,654 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-227-0 01:50:45,655 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-227-0 ready 01:50:45,655 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-227-0 01:50:45,655 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 validated 01:50:45,655 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:45,655 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-227-0: success 01:50:45,655 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-227-0 01:50:45,655 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-227-0 01:50:45,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 01:50:45,655 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 candidate ready 01:50:45,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:50:45,655 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-227-0: canCommit - candidate rootPath: / 01:50:45,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:45,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:50:45,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:50:45,655 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-227-0: canCommit - messages: [] 01:50:45,655 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-227-0: preCommit - successfulFromPrevious: [] 01:50:45,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 1 01:50:45,655 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 prepared 01:50:45,655 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:45,655 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-227-0 01:50:45,655 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:45,655 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-227-0 01:50:45,655 PM [cluster-test-shard-dispatcher-30] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 197 01:50:45,656 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:50:45,656 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:50:45,656 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-227-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:45,656 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:45,656 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,656 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 submitted to persistence 01:50:45,656 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 0, nextIndex: 1 01:50:45,656 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:45,656 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,656 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 01:50:45,657 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:50:45,657 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:50:45,657 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader status sync done true 01:50:45,657 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:45,657 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 0 01:50:45,657 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from -1 to 0 01:50:45,657 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:50:45,657 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 01:50:45,657 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 01:50:45,657 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 01:50:45,657 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:50:45,657 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:45,657 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, nanosAgo=3912286} is already disabled 01:50:45,657 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log Simple{size=197} 01:50:45,657 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:50:45,657 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 01:50:45,657 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 2 01:50:45,657 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:45,657 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:50:45,657 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:50:45,658 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 1, nextIndex: 2 01:50:45,658 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:45,658 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 1 01:50:45,658 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 0 to 1 01:50:45,658 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 01:50:45,658 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-227-0 01:50:45,658 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-227-0: commit - successfulFromPrevious: [] 01:50:45,658 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 changing state from COMMIT_PENDING to COMMITTED 01:50:45,658 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-227-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/system/testProbe-762#-653241882] 01:50:45,658 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#1713400811] with name member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher 01:50:45,658 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#1713400811] 01:50:45,658 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 01:50:45,659 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 01:50:45,659 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 156.9 μs 01:50:45,659 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:45,659 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 01:50:45,659 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#1713400811] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:45,660 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 01:50:45,660 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-944224862] 01:50:45,660 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:45,660 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 Queuing TransactionOperation 01:50:45,660 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-926884118, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@76941517] 01:50:45,660 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-926884118, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@76941517] 01:50:45,660 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 1 01:50:45,660 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 01:50:45,660 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 0 01:50:45,660 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-926884118)] for shard cars 01:50:45,660 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from -1 to 0 01:50:45,660 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 01:50:45,660 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:50:45,660 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@2b2fc494 01:50:45,660 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 01:50:45,660 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 01:50:45,660 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 1 01:50:45,660 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 0 to 1 01:50:45,660 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 01:50:45,661 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-227-0 01:50:45,661 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-77710130] with name member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher 01:50:45,661 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 01:50:45,661 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 01:50:45,661 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: handleForwardedReadyTransaction for member-1-ShardTransactionTest-fe-0-txn-228-0 01:50:45,661 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,661 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader)] 01:50:45,661 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-77710130] 01:50:45,661 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 01:50:45,662 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-77710130] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:45,662 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 163.1 μs 01:50:45,663 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-228-0 01:50:45,663 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-228-0 of 1 operations, client version 13 01:50:45,663 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-228-0 01:50:45,663 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-228-0 ready 01:50:45,664 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0 canCommit 01:50:45,664 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-926884118)] resolved 01:50:45,664 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: successfully resolved all cohort actors 01:50:45,664 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0 finishCanCommit 01:50:45,665 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: sending CanCommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-228-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-926884118)] 01:50:45,665 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-926884118)] 01:50:45,665 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Can committing transaction member-1-ShardTransactionTest-fe-0-txn-228-0 01:50:45,665 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-228-0 01:50:45,665 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 validated 01:50:45,665 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:45,665 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-228-0: success 01:50:45,665 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:50:45,665 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: canCommit returning result: true 01:50:45,665 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0 commit 01:50:45,665 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0 finish commit 01:50:45,665 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: Sending CommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-228-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-926884118)] 01:50:45,665 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-926884118)] 01:50:45,665 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-228-0 01:50:45,665 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-228-0 01:50:45,665 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 candidate ready 01:50:45,665 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-228-0: canCommit - candidate rootPath: / 01:50:45,666 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-228-0: canCommit - messages: [] 01:50:45,666 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-228-0: preCommit - successfulFromPrevious: [] 01:50:45,666 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 prepared 01:50:45,666 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:45,666 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-228-0 01:50:45,666 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-228-0 01:50:45,666 PM [cluster-test-shard-dispatcher-30] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 208 01:50:45,666 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:50:45,666 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:50:45,666 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-228-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:45,666 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:50:45,666 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,666 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 01:50:45,666 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 submitted to persistence 01:50:45,666 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:50:45,668 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 01:50:45,668 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:50:45,668 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:50:45,668 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log Simple{size=208} 01:50:45,668 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:50:45,668 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 2 01:50:45,668 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:45,668 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 2, nextIndex: 3 01:50:45,668 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:50:45,668 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:50:45,668 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 2 01:50:45,668 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 1 to 2 01:50:45,668 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 01:50:45,668 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-228-0 01:50:45,668 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-228-0: commit - successfulFromPrevious: [] 01:50:45,668 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 changing state from COMMIT_PENDING to COMMITTED 01:50:45,668 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-228-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testForwardedReadyTransactionForwardedToLeader_member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader%23-926884118$a] 01:50:45,669 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 01:50:45,669 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:45,669 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 01:50:45,669 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-228-0: commit succeeded 01:50:45,669 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0 01:50:45,669 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 01:50:45,669 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-926884118)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:45,669 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@2b2fc494 01:50:45,669 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:45,669 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 01:50:45,669 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 138.7 μs 01:50:45,670 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Issuing gracefulStop to shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:45,670 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader - waiting on 1 shards 01:50:45,670 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onShutDown 01:50:45,670 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Initiating leader transfer 01:50:45,670 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 01:50:45,670 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@a240c74 01:50:45,670 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@a240c74 01:50:45,670 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Attempting to transfer leadership 01:50:45,670 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:45,671 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 2 01:50:45,671 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:50:45,671 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 2 01:50:45,671 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 1 to 2 01:50:45,671 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 01:50:45,671 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-228-0 01:50:45,670 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:45,671 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:45,671 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 01:50:45,671 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 01:50:45,671 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 01:50:45,672 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 62.61 μs 01:50:45,672 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader] 01:50:45,672 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, matchIndex: 2, lastIndex: 2, isVoting: true 01:50:45,672 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Follower's log matches - sending ElectionTimeout 01:50:45,672 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:45,672 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 01:50:45,672 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Leader transfer complete 01:50:45,672 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:45,672 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:46,85 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 01:50:46,86 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-926884118]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:46,86 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:46,87 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader)_slicer-id-532: Closing 01:50:46,87 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Aborting 0 pending queued transactions 01:50:46,87 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 01:50:46,87 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 01:50:46,87 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 01:50:46,87 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: shutting down 01:50:46,88 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 01:50:46,88 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 01:50:46,88 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:46,88 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:46,94 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:46,94 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:46,95 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:46,95 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:46,97 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:46,107 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:50:46,107 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Closing 01:50:46,107 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Aborting 0 pending queued transactions 01:50:46,107 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 01:50:46,108 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 01:50:46,109 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:46,109 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:47,107 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [1030504184704156361] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:47,124 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:50:47.127] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config [WARN] [03/11/2024 13:50:47.124] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-836)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:47,130 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-8171308151512354190] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:47,142 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:47,142 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/11/2024 13:50:47.145] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:47,177 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:50:47,183 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-1314275778611372565] 01:50:47,183 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:47,185 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:47,185 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:47,197 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:47,197 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:47,197 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:47,197 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:47,197 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:47,207 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:47,215 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [8590567919541876812] 01:50:47,216 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:47,217 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:47,217 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:47,219 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:47,219 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:47,238 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:47,262 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2380544311794286050] 01:50:47,262 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:47,269 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:47,269 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:47,269 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:47,269 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:47,274 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:47,275 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:47,275 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:47,276 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithSingleShard 01:50:47,280 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:47,283 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 01:50:47,283 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-728302873] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:47,284 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: no initial generation override, starting from 0 01:50:47,284 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:50:47,284 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5b375aa7 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@796cca74 01:50:47,285 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7c24a0a0 01:50:47,296 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, 0, 1710165047296, None)) 01:50:47,296 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithSingleShard 01:50:47,296 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Creating local shard: member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:47,296 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:47,296 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:47,298 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:47,301 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:47,302 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165047295,0,0)) 01:50:47,302 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5b6eed1a 01:50:47,302 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@796cca74 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@720371d7 01:50:47,302 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} started 01:50:47,302 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:47,304 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:47,306 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithSingleShard 01:50:47,306 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:47,306 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:47,307 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:47,307 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:47,307 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: updated peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard on shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1716963288] 01:50:47,307 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testChainedTransactionFailureWithSingleShard, persistent : true 01:50:47,308 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testChainedTransactionFailureWithSingleShard with journal batch size 1 01:50:47,309 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1716963288] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:47,309 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-1823502181 created and ready for shard:member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:47,309 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-1:snapshot-read#-1840820416] 01:50:47,309 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-1823502181] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:47,309 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-1:snapshot-read#-1840820416] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:47,310 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-582285908] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:47,313 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:47,313 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:47,313 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Initializing shard [cars] 01:50:47,313 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is initialized 01:50:47,313 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:47,313 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 01:50:47,313 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from null to Follower 01:50:47,313 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard 01:50:47,313 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from null to Follower 01:50:47,317 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 01:50:47,319 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: no initial generation override, starting from 0 01:50:47,320 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:50:47,320 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@45081f6f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3ea2fc3a 01:50:47,320 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@607f9341 01:50:47,320 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithSingleShard 01:50:47,320 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Creating local shard: member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:47,320 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, 0, 1710165047320, None)) 01:50:47,321 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:47,321 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:47,321 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165047319,0,0)) 01:50:47,321 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4aa9a182 01:50:47,321 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3ea2fc3a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@50a9b846 01:50:47,321 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} started 01:50:47,322 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:47,322 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:47,322 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:47,322 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:47,322 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: updated peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard on shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard#163216160] 01:50:47,322 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testChainedTransactionFailureWithSingleShard, persistent : true 01:50:47,322 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard#163216160] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:47,323 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-1219037835 created and ready for shard:member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:47,323 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-1219037835] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:47,323 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-2:snapshot-read#1527706794] 01:50:47,323 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testChainedTransactionFailureWithSingleShard with journal batch size 1 01:50:47,323 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-2:snapshot-read#1527706794] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:47,323 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:47,323 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:47,324 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Initializing shard [cars] 01:50:47,324 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard is initialized 01:50:47,324 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:47,324 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 01:50:47,324 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:47,324 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from null to Follower 01:50:47,324 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard 01:50:47,324 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-2-shard-cars-testChainedTransactionFailureWithSingleShard from null to Follower 01:50:47,372 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-728302873] 01:50:47,372 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: findLocalShard : cars 01:50:47,372 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1716963288] 01:50:47,594 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:47,595 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 01:50:47,595 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testChainedTransactionFailureWithSingleShard] 01:50:47,595 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:47,595 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 1 01:50:47,595 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:47,595 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Closing 01:50:47,595 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:50:47,595 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 01:50:47,595 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from Follower to Candidate 01:50:47,595 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from Follower to Candidate 01:50:47,825 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 01:50:47,825 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:47,825 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 2 01:50:47,848 PM [cluster-test-shard-dispatcher-25] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:48,105 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 01:50:48,105 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:48,105 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 3 01:50:48,207 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:48,207 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:48,207 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:48,207 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:48,207 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:48,207 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:48,207 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:48,207 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:48,207 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:48,215 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:48,215 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:48,215 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:48,215 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:48,215 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:48,215 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:48,252 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 closeTransaction called 01:50:48,252 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_30#1750808237)] 01:50:48,252 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 closeTransaction called 01:50:48,252 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_32#1217732533)] 01:50:48,282 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:48,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 01:50:48,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:48,385 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 4 01:50:48,385 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:48,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:48,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Cluster state: [] 01:50:48,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:48,388 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:48,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=null 01:50:48,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:48,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:48,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:48,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:48,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:48,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testChainedTransactionFailureWithSingleShard] 01:50:48,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:48,390 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:48,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:50:48,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: became leader with frontend state for [] 01:50:48,390 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from Candidate to Leader 01:50:48,390 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderPayloadVersion=13] 01:50:48,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:48,390 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:48,390 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from Candidate to Leader 01:50:48,390 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All Shards are ready - data store testChainedTransactionFailureWithSingleShard is ready 01:50:48,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Closing 01:50:48,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testChainedTransactionFailureWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:48,391 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithSingleShard status sync done false 01:50:48,391 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderPayloadVersion=13] 01:50:48,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:48,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:48,391 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All Shards are ready - data store testChainedTransactionFailureWithSingleShard is ready 01:50:48,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: -1, nextIndex: 0 01:50:48,393 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:48,437 PM [main] [DEBUG] AbstractDOMBroker - Transaction chain 0 created with listener Mock for DOMTransactionChainListener, hashCode: 605292557, backing store chains {CONFIGURATION=org.opendaylight.controller.cluster.datastore.TransactionChainProxy@5da1fde3} 01:50:48,438 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testChainedTransactionFailureWithSingleShard commit rateLimit adjusted to 80.58185946995822 01:50:48,438 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 01:50:48,438 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-582285908] 01:50:48,438 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 01:50:48,438 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:48,438 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard, primaryVersion=13] 01:50:48,439 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard, primaryVersion=13] 01:50:48,439 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard)] for shard cars 01:50:48,439 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard)] found - creating WRITE_ONLY transaction context 01:50:48,439 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Readying 1 components for commit 01:50:48,439 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:48,439 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 01:50:48,439 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 directCommit called 01:50:48,439 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 sending 1 batched modifications, ready: true 01:50:48,439 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard)] 01:50:48,439 PM [main] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 is submitted for execution. 01:50:48,439 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 canCommit 01:50:48,441 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, disabling transaction tracking 01:50:48,441 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 01:50:48,441 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 01:50:48,441 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:48,441 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:48,441 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:48,442 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}]]] 01:50:48,442 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 01:50:48,442 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 01:50:48,442 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:50:48,442 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0} 01:50:48,442 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@7a1101a6 01:50:48,442 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying 1 batched modifications for Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 01:50:48,442 PM [cluster-test-shard-dispatcher-21] [ERROR] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Error handling BatchedModifications for Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:48,443 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 01:50:48,443 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 01:50:48,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}]]] 01:50:48,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Number of entries to be appended = 1 01:50:48,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:48,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71} 01:50:48,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 01:50:48,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Log size is now 1 01:50:48,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:48,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 01:50:48,452 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithSingleShard status sync done true 01:50:48,453 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 modifications failed org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:48,453 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during phase CAN_COMMIT, starting Abort org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:48,454 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DOMBrokerTransactionChain - Transaction chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@78549223 failed. TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema.]]} at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1119) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) at org.opendaylight.controller.cluster.datastore.SingleCommitCohortProxy$1.onComplete(SingleCommitCohortProxy.java:60) at akka.dispatch.OnComplete.internal(Future.scala:299) at akka.dispatch.OnComplete.internal(Future.scala:297) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:50:48,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: 0, nextIndex: 1 01:50:48,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:48,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Setting last applied to 0 01:50:48,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from -1 to 0 01:50:48,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71} 01:50:48,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:50:48,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} does not match any client, pre-disabling client. 01:50:48,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, nanosAgo=13114043} is already disabled 01:50:48,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:48,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:48,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}]]] 01:50:48,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=0 01:50:48,455 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 aborted successfully 01:50:48,459 PM [main] [DEBUG] DOMBrokerTransactionChain - Chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@78549223 is no longer running 01:50:48,460 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:50:48,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:48,460 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithSingleShard - waiting on 1 shards 01:50:48,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onShutDown 01:50:48,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Initiating leader transfer 01:50:48,460 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:50:48,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:48,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:48,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@ed5e596 01:50:48,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@ed5e596 01:50:48,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}]]] 01:50:48,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Attempting to transfer leadership 01:50:48,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Number of entries to be appended = 1 01:50:48,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:48,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73} 01:50:48,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:48,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 01:50:48,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:48,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Log size is now 2 01:50:48,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Commit index set to 0 01:50:48,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:48,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:50:48,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Setting last applied to 0 01:50:48,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from -1 to 0 01:50:48,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71} 01:50:48,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:50:48,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} does not match any client, pre-disabling client. 01:50:48,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 01:50:48,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=0 01:50:48,462 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard] 01:50:48,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: 1, nextIndex: 2 01:50:48,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:48,462 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:50:48,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Setting last applied to 1 01:50:48,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from 0 to 1 01:50:48,462 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:48,462 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:48,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73} 01:50:48,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0} has no callback 01:50:48,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: fakeSnapshot purging log to 0 for term 4 01:50:48,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:48,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testChainedTransactionFailureWithSingleShard, matchIndex: 1, lastIndex: 1, isVoting: true 01:50:48,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 01:50:48,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:48,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Leader transfer complete 01:50:48,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=1 01:50:48,464 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Commit index set to 1 01:50:48,464 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:48,464 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Setting last applied to 1 01:50:48,464 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from 0 to 1 01:50:48,464 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73} 01:50:48,464 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: fakeSnapshot purging log to 0 for term 4 01:50:48,464 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=1 01:50:48,875 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All shards shutdown - sending PoisonPill to self 01:50:48,875 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1716963288]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:48,875 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:48,875 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader)_slicer-id-537: Closing 01:50:48,875 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Aborting 0 pending queued transactions 01:50:48,875 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 01:50:48,876 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 01:50:48,876 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:50:48,876 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: shutting down 01:50:48,876 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 01:50:48,876 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 01:50:48,876 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#95469614] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#95469614] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:48,876 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-728302873] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-728302873] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:48,876 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#95469614] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#95469614] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:48,877 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:48,877 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:48,882 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:48,882 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:48,883 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:48,883 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:48,886 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:48,898 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 01:50:48,898 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:50:48,898 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Closing 01:50:48,898 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Aborting 0 pending queued transactions 01:50:48,898 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 01:50:48,899 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:48,899 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:49,891 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-1314275778611372565] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:49,921 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:49,921 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [8590567919541876812] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:49,921 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:50:49.925] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:49,930 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:49,931 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/11/2024 13:50:49.934] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:49,962 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:50:49,969 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4531921717197286301] 01:50:49,970 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:49,972 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:49,972 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:49,972 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:49,972 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:49,983 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:49,984 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:49,984 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:49,987 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:50:49,994 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [8412456408546357214] 01:50:49,994 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:49,995 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:49,995 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:50,3 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:50,3 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:50,13 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:50,19 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-8177704271766087417] 01:50:50,19 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:50,21 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:50,21 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:50,22 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:50,22 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:50,34 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:50,35 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:50,37 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithShardLeaderNotResponding 01:50:50,49 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:50,49 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 01:50:50,51 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#1591124857] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:50,59 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:50,60 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithShardLeaderNotResponding 01:50:50,60 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Creating local shard: member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,60 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:50,61 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,61 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: no initial generation override, starting from 0 01:50:50,61 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:50:50,61 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3da2a09 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2394f7a2 01:50:50,61 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@66967db4 01:50:50,61 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, 0, 1710165050061, None)) 01:50:50,61 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165050060,0,0)) 01:50:50,62 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2394f7a2 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4c219484 01:50:50,62 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} started 01:50:50,63 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:50,63 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@b77fab0 01:50:50,63 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:50,63 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,63 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,63 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,63 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: updated peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding on shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-712718994] 01:50:50,64 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithShardLeaderNotResponding, persistent : true 01:50:50,64 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithShardLeaderNotResponding with journal batch size 1 01:50:50,65 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:50,67 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithShardLeaderNotResponding 01:50:50,67 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-712718994] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:50,67 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-1:snapshot-read#-1535983807] 01:50:50,67 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-1650347354 created and ready for shard:member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,67 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-1:snapshot-read#-1535983807] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:50,67 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-1650347354] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:50,69 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:50,69 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:50,69 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 01:50:50,69 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 01:50:50,69 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from null to Follower 01:50:50,69 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Initializing shard [cars] 01:50:50,69 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is initialized 01:50:50,69 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , registered listener akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding 01:50:50,69 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from null to Follower 01:50:50,73 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 01:50:50,74 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#137151604] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:50,76 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: no initial generation override, starting from 0 01:50:50,76 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:50:50,76 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@9c902a3 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1e5a03d6 01:50:50,76 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4841e801 01:50:50,76 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithShardLeaderNotResponding 01:50:50,77 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Creating local shard: member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,77 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:50,77 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,77 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, 0, 1710165050076, None)) 01:50:50,77 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165050075,0,0)) 01:50:50,77 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7fb59cd9 01:50:50,77 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1e5a03d6 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@72fdb834 01:50:50,78 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} started 01:50:50,78 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:50,78 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,78 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,78 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,78 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: updated peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding on shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding#-401702424] 01:50:50,79 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding#-401702424] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:50,79 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithShardLeaderNotResponding, persistent : true 01:50:50,79 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-516600588 created and ready for shard:member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,79 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-2:snapshot-read#985681597] 01:50:50,79 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-516600588] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:50,80 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-2:snapshot-read#985681597] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:50,80 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithShardLeaderNotResponding with journal batch size 1 01:50:50,80 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:50,80 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:50,80 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Initializing shard [cars] 01:50:50,80 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding is initialized 01:50:50,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 01:50:50,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 01:50:50,81 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,81 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from null to Follower 01:50:50,81 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithShardLeaderNotResponding , registered listener akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding 01:50:50,81 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-2-shard-cars-testTransactionWithShardLeaderNotResponding from null to Follower 01:50:50,97 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:50,102 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:50,128 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#1591124857] 01:50:50,128 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: findLocalShard : cars 01:50:50,128 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-712718994] 01:50:50,309 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:50,309 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Received ElectionTimeout - switching to Candidate 01:50:50,309 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithShardLeaderNotResponding] 01:50:50,309 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,309 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 1 01:50:50,309 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:50,309 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Closing 01:50:50,309 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 01:50:50,309 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 01:50:50,310 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from Follower to Candidate 01:50:50,310 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from Follower to Candidate 01:50:50,618 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 01:50:50,618 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,618 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 2 01:50:50,908 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 01:50:50,908 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,908 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 3 01:50:50,989 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:50,989 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:50,990 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:50,990 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,990 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,990 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:50,990 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,990 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,990 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,996 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:50,996 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,996 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,996 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:50,996 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:50,996 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:51,39 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:51,178 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 01:50:51,178 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:51,178 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 4 01:50:51,178 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:51,180 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:51,180 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Cluster state: [] 01:50:51,180 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:51,180 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:51,180 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=null 01:50:51,181 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:51,181 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:51,181 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:51,182 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:51,182 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:51,182 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithShardLeaderNotResponding] 01:50:51,182 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:51,182 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:51,182 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: true, hasLeader : true 01:50:51,183 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: became leader with frontend state for [] 01:50:51,183 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from Candidate to Leader 01:50:51,183 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderPayloadVersion=13] 01:50:51,183 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:51,183 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:51,183 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from Candidate to Leader 01:50:51,183 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 01:50:51,184 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Closing 01:50:51,184 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithShardLeaderNotResponding peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:51,184 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithShardLeaderNotResponding, leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderPayloadVersion=13] 01:50:51,184 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:51,184 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:51,184 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 01:50:51,184 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding Received follower initial sync status for member-2-shard-cars-testTransactionWithShardLeaderNotResponding status sync done false 01:50:51,184 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithShardLeaderNotResponding updated: matchIndex: -1, nextIndex: 0 01:50:51,185 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:51,185 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 01:50:51,185 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#137151604] 01:50:51,186 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:51,186 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 Queuing TransactionOperation 01:50:51,186 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding, primaryVersion=13] 01:50:51,186 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding, primaryVersion=13] 01:50:51,186 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] for shard cars 01:50:51,186 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 01:50:51,186 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 01:50:51,188 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, disabling transaction tracking 01:50:51,188 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 01:50:51,188 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 01:50:51,188 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:51,188 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:51,188 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:51,188 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}]]] 01:50:51,188 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Creating transaction : member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 01:50:51,188 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_33#1154807177] 01:50:51,189 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_33#1154807177] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:51,189 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 01:50:51,190 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}]]] 01:50:51,190 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_33#1154807177, transactionId=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0, version=13] 01:50:51,190 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Number of entries to be appended = 1 01:50:51,190 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:51,190 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70} 01:50:51,190 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 01:50:51,190 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:51,190 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_33#1154807177)] 01:50:51,190 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Log size is now 1 01:50:51,190 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithShardLeaderNotResponding, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:51,190 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 01:50:51,191 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding Received follower initial sync status for member-2-shard-cars-testTransactionWithShardLeaderNotResponding status sync done true 01:50:51,191 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithShardLeaderNotResponding updated: matchIndex: 0, nextIndex: 1 01:50:51,191 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:51,191 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Setting last applied to 0 01:50:51,191 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Moving last applied index from -1 to 0 01:50:51,191 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70} 01:50:51,191 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:50:51,191 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} does not match any client, pre-disabling client. 01:50:51,191 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, nanosAgo=3781463} is already disabled 01:50:51,191 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:51,192 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Persisting ApplyJournalEntries with index=0 01:50:51,192 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 ReadData operation succeeded 01:50:51,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Commit index set to 0 01:50:51,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:50:51,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Setting last applied to 0 01:50:51,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Moving last applied index from -1 to 0 01:50:51,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70} 01:50:51,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:50:51,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} does not match any client, pre-disabling client. 01:50:51,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Persisting ApplyJournalEntries with index=0 01:50:51,193 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 01:50:51,193 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:51,193 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader)_slicer-id-542: Closing 01:50:51,193 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Aborting 0 pending queued transactions 01:50:51,193 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 01:50:51,194 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:51,194 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:51,204 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:51,204 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:51,205 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:51,205 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:51,206 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:51,221 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithShardLeaderNotResponding 01:50:51,221 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionWithShardLeaderNotResponding commit rateLimit adjusted to 80.58185946995822 01:50:51,221 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:51,221 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 01:50:51,222 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:51,222 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:51,222 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:50:51,222 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 01:50:51,222 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 01:50:51,222 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Queuing TransactionOperation 01:50:51,223 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Readying 1 components for commit 01:50:51,223 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Readying transaction for shard cars 01:50:51,223 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Queuing TransactionOperation 01:50:51,223 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 canCommit 01:50:51,286 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 01:50:51,315 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 01:50:51,315 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 01:50:51,315 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 01:50:51,386 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 01:50:51,415 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 01:50:51,415 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 01:50:51,415 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 01:50:51,485 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 01:50:51,515 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 01:50:51,516 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 01:50:51,516 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 01:50:51,586 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 01:50:51,615 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 01:50:51,615 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 01:50:51,615 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 01:50:51,686 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Creating NoOpTransaction because of error akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:51,687 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] NoOpTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:51,687 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] NoOpTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 directCommit called, failure org.opendaylight.controller.cluster.datastore.exceptions.ShardLeaderNotRespondingException: Could not create a READ_WRITE transaction on shard cars. The shard leader isn't responding. at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport.createTransactionContext(RemoteTransactionContextSupport.java:213) at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport.onCreateTransactionComplete(RemoteTransactionContextSupport.java:195) at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport$1.onComplete(RemoteTransactionContextSupport.java:136) at akka.dispatch.OnComplete.internal(Future.scala:299) at akka.dispatch.OnComplete.internal(Future.scala:297) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:50:51,688 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:50:51,688 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithShardLeaderNotResponding data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:50:51,688 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 01:50:51,688 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:50:51,689 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithShardLeaderNotResponding data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:50:51,689 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 01:50:51,690 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 01:50:51,690 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:50:51,690 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Closing 01:50:51,690 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Aborting 0 pending queued transactions 01:50:51,690 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 01:50:51,691 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:51,691 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:52,208 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4531921717197286301] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:52,708 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:52,708 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:52,709 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [8412456408546357214] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:52,711 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:52.713] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:52,728 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:52,728 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:52,731 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:52.733] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:52,760 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:52,767 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-4849018227947970292] 01:50:52,767 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:52,769 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:52,769 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:52,769 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:52,769 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:52,781 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:52,781 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:52,781 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:52,785 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:52,792 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [9200127487165592706] 01:50:52,792 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:52,794 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:52,794 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:52,804 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:52,804 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:52,810 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:52,816 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2845539625858783376] 01:50:52,817 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:52,818 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:52,818 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:52,829 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:52,829 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:52,829 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:52,831 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:52,833 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 01:50:52,838 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#434701654] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:52,843 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: no initial generation override, starting from 0 01:50:52,843 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:50:52,843 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@ee443f1 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6ceecb41 01:50:52,843 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@49d63c57 01:50:52,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1710165052844, None)) 01:50:52,844 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165052843,0,0)) 01:50:52,845 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6ceecb41 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7b9d9659 01:50:52,845 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} started 01:50:52,846 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:52,848 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:52,850 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 01:50:52,850 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:50:52,858 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:52,861 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1555179585] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:52,861 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:52,861 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:50:52,863 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:52,864 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: no initial generation override, starting from 0 01:50:52,864 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:50:52,864 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@59ade199 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@65194262 01:50:52,864 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@201bd0a5 01:50:52,864 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1710165052864, None)) 01:50:52,865 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165052863,0,0)) 01:50:52,865 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 01:50:52,865 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:52,866 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@65194262 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3ed44e6d 01:50:52,866 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} started 01:50:52,866 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@61cce0d4 01:50:52,866 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:52,866 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:52,866 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:52,866 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:52,866 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:52,867 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 01:50:52,867 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#227541588] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:52,867 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-214787641 created and ready for shard:member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:52,867 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-976417904] 01:50:52,867 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-214787641] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:52,867 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 01:50:52,867 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-976417904] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:52,868 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:52,868 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:52,868 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:50:52,868 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:52,868 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 01:50:52,868 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 01:50:52,868 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 01:50:52,868 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 01:50:52,868 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 01:50:52,868 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 01:50:52,868 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:52,869 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:52,869 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:52,869 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7cdee80c 01:50:52,869 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:52,869 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:52,869 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:52,871 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:52,871 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:52,871 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: updated peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1589003230] 01:50:52,871 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 01:50:52,871 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1589003230] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:52,872 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 01:50:52,873 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-1446315638 created and ready for shard:member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:52,873 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-2:snapshot-read#-965348862] 01:50:52,873 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-1446315638] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:52,873 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-2:snapshot-read#-965348862] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:52,874 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:52,874 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:52,874 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:50:52,874 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:52,875 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 01:50:52,875 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:52,875 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 01:50:52,875 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 01:50:52,875 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 01:50:52,875 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 01:50:52,916 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#434701654] 01:50:52,917 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 01:50:52,917 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#227541588] 01:50:53,156 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:53,156 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 01:50:53,156 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:50:53,156 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,156 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 1 01:50:53,156 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:53,156 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:50:53,156 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:50:53,156 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:53,156 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 01:50:53,156 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 01:50:53,476 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:53,476 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,476 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 2 01:50:53,482 PM [cluster-test-shard-dispatcher-26] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:53,787 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:53,787 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:53,787 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:53,787 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,787 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,787 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:53,787 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,787 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,787 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,792 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:53,792 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,792 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,792 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:53,792 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,792 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,795 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:53,795 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,796 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 3 01:50:53,796 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,796 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:53,797 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:50:53,797 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:53,797 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:53,797 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=null 01:50:53,797 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:53,797 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,797 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 01:50:53,800 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 01:50:53,800 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 01:50:53,800 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:50:53,800 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:53,800 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 01:50:53,800 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 01:50:53,800 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [] 01:50:53,800 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Leader 01:50:53,800 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 01:50:53,800 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:53,800 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:53,800 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Leader 01:50:53,800 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 01:50:53,801 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:53,801 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,801 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 01:50:53,801 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 01:50:53,801 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:53,801 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:53,801 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 01:50:53,802 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: -1, nextIndex: 0 01:50:53,802 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:53,825 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testSingleShardTransactionsWithLeaderChanges commit rateLimit adjusted to 80.58185946995822 01:50:53,825 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:50:53,826 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1555179585] 01:50:53,826 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Queuing TransactionOperation 01:50:53,826 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Queuing TransactionOperation 01:50:53,826 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Readying 1 components for commit 01:50:53,826 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Readying transaction for shard cars 01:50:53,826 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Queuing TransactionOperation 01:50:53,826 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 canCommit 01:50:53,827 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:53,827 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 01:50:53,827 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 01:50:53,827 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges)] for shard cars 01:50:53,827 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges)] found - creating WRITE_ONLY transaction context 01:50:53,827 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:50:53,827 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 01:50:53,827 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 directCommit called 01:50:53,827 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 sending 2 batched modifications, ready: true 01:50:53,827 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges)] 01:50:53,830 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, disabling transaction tracking 01:50:53,830 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 01:50:53,830 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 01:50:53,830 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:53,830 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:53,830 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,830 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}]]] 01:50:53,830 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying 2 batched modifications for Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Readying Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 of 2 operations, client version 13 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: readying transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 ready 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Validating transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 validated 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: canCommit for member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: success 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Preparing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 candidate ready 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: canCommit - messages: [] 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 prepared 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Finishing commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Starting commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 181 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=181}] 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=181}] 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,831 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 submitted to persistence 01:50:53,832 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}]]] 01:50:53,832 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 01:50:53,832 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:53,833 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71} 01:50:53,833 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 01:50:53,833 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 1 01:50:53,833 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:53,833 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 01:50:53,833 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 01:50:53,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 01:50:53,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=181}] 01:50:53,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 0, nextIndex: 1 01:50:53,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:50:53,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 0 01:50:53,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 01:50:53,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71} 01:50:53,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:50:53,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} does not match any client, pre-disabling client. 01:50:53,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, nanosAgo=4644044} is already disabled 01:50:53,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:53,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,834 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=181}]]] 01:50:53,835 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=0 01:50:53,837 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=181}]]] 01:50:53,837 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 01:50:53,837 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:50:53,837 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log Simple{size=181} 01:50:53,837 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=181}] 01:50:53,837 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 2 01:50:53,837 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 0 01:50:53,837 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:53,837 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:50:53,837 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 0 01:50:53,837 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 01:50:53,837 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71} 01:50:53,837 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:50:53,837 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} does not match any client, pre-disabling client. 01:50:53,838 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=181}] 01:50:53,838 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=0 01:50:53,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 1, nextIndex: 2 01:50:53,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:50:53,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 1 01:50:53,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 01:50:53,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 1 data Simple{size=181} 01:50:53,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Resuming commit of transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:50:53,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:50:53,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:50:53,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleShardTransactionsWithLeaderChanges_member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges$a] 01:50:53,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-251308212] with name member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 01:50:53,841 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 0 for term 3 01:50:53,842 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:53,842 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=1 01:50:53,842 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-251308212] 01:50:53,842 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 01:50:53,842 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-251308212] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:53,842 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 95.96 μs 01:50:53,845 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 successfully completed direct commit 01:50:53,845 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 modifications completed with CommitTransactionReply [version=13] 01:50:53,845 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testSingleShardTransactionsWithLeaderChanges 01:50:53,846 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 1 01:50:53,846 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:50:53,846 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 1 01:50:53,846 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 01:50:53,846 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 1 data Simple{size=181} 01:50:53,846 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying foreign transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:50:53,847 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1608063244] with name member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 01:50:53,847 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 0 for term 3 01:50:53,847 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=1 01:50:53,847 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:53,848 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,848 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 01:50:53,848 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Re-initializing to Follower with previous leaderId member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,848 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:50:53,848 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : true 01:50:53,848 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:53,848 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,848 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-547: Closing 01:50:53,848 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1608063244] 01:50:53,848 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 01:50:53,848 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 01:50:53,849 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 137.2 μs 01:50:53,849 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1608063244] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:53,849 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:50:53,849 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:53,850 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:53,850 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:53,861 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:53,861 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:53,861 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:53,861 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:53,863 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:50:53,877 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 01:50:53,927 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1555179585] 01:50:53,927 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 01:50:53,927 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1589003230] 01:50:53,928 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,978 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,991 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 01:50:53,991 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 01:50:53,992 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 01:50:53,992 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 01:50:53,992 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:50:53,992 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,992 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 4 01:50:53,992 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:53,992 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 4 01:50:53,992 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:50:53,992 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:50:53,993 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:53,993 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:53,993 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 01:50:53,993 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=-1] 01:50:53,993 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:53,993 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:53,993 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 01:50:54,38 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:50:54,46 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [6027569023003710967] 01:50:54,46 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:54,48 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:54,48 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:54,48 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:54,49 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:54,59 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:54,61 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:54,63 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 01:50:54,67 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-852219136] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:54,68 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:50:54,71 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: recovered identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:50:54,72 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 01:50:54,72 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} 01:50:54,72 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,72 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3e21b692 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@a36772c 01:50:54,72 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@35ccb4b6 01:50:54,74 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1710165054074, None)) 01:50:54,76 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165054073,0,0)) 01:50:54,77 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@a36772c to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@d224880 01:50:54,77 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@340496c7 01:50:54,77 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} started 01:50:54,77 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:54,77 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,78 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 01:50:54,79 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#1936383051] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:54,79 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-2014100098] 01:50:54,79 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 01:50:54,79 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-1511703857] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:54,80 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-1511703857 created and ready for shard:member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,81 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-2014100098] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:54,83 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:50:54,83 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,83 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:50:54,83 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,83 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:50:54,83 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,83 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 01:50:54,83 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received ReplicatedLogEntry for recovery: index: 0, size: 71 01:50:54,83 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 01:50:54,83 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received ReplicatedLogEntry for recovery: index: 1, size: 181 01:50:54,83 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 01:50:54,83 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received apply journal entries for recovery, applying to state: 0 to 0 01:50:54,83 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: starting log recovery batch with max size 1 01:50:54,83 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: ignoring unhandled payload DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71} 01:50:54,83 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 01:50:54,83 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 01:50:54,83 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received apply journal entries for recovery, applying to state: 1 to 1 01:50:54,83 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: starting log recovery batch with max size 1 01:50:54,84 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4121d091}, purged=MutableUnsignedLongSet{size=0}} 01:50:54,84 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:50:54,84 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 01:50:54,84 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:54,84 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed in 771.0 μs - Switching actor to Follower - last log index = 1, last log term = 3, snapshot index = -1, snapshot term = -1, journal size = 2 01:50:54,85 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:50:54,85 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:54,85 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 01:50:54,85 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 01:50:54,85 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 01:50:54,85 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 01:50:54,85 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 01:50:54,127 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1555179585] 01:50:54,128 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 01:50:54,128 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1589003230] 01:50:54,201 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:54,201 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,202 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 5 01:50:54,202 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,372 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:54,372 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,372 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 6 01:50:54,373 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,552 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:54,552 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,552 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 7 01:50:54,552 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,683 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(akka://cluster-test@127.0.0.1:2558, Removed) 01:50:54,683 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 01:50:54,684 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:50:54,684 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,684 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 4 01:50:54,684 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 4 01:50:54,684 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:50:54,684 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:50:54,684 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:54,684 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 01:50:54,684 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 01:50:54,721 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:54,721 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,721 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 8 01:50:54,722 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,864 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [-4849018227947970292] but current is [6027569023003710967]. Reason: ActorSystem terminated 01:50:54,869 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-4849018227947970292] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:54,882 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:50:54,882 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,882 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 9 01:50:54,882 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,918 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2558, Leaving)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 01:50:54,919 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking unreachable node [akka://cluster-test@127.0.0.1:2558] as [Down] 01:50:54,921 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:54,937 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=4, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3], my term is 4 01:50:54,937 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=4, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3] - currentTerm: 4, votedFor: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastIndex: 1, lastTerm: 3 01:50:54,937 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 01:50:54,937 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3], my term is 4 01:50:54,937 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3]" message is greater than Candidate's term 4 - switching to Follower 01:50:54,937 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=null 01:50:54,937 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 3 01:50:54,937 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,937 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:50:54,937 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 5 01:50:54,937 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:50:54,937 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:50:54,937 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 01:50:54,937 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 01:50:54,938 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=false], my term is 9 01:50:54,938 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=false], current voteCount: 1 01:50:54,939 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 9 01:50:54,939 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:50:54,939 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:50:54,939 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:54,939 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Leader, election term: 9 01:50:54,939 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 01:50:54,939 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges] 01:50:54,939 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Leader 01:50:54,939 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 01:50:54,939 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:54,939 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:54,939 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Leader 01:50:54,939 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 01:50:54,939 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:54,939 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:50:54,939 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3] with higher term appears reachable, updating term. 01:50:54,939 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 6 in "RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3]" message is greater than follower's term 5 - updating term 01:50:54,939 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=null 01:50:54,940 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3] - currentTerm: 6, votedFor: null, lastIndex: 1, lastTerm: 3 01:50:54,940 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,940 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 01:50:54,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:54,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:50:54,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3] with higher term appears reachable, updating term. 01:50:54,941 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 7 in "RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3]" message is greater than follower's term 6 - updating term 01:50:54,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=null 01:50:54,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3] - currentTerm: 7, votedFor: null, lastIndex: 1, lastTerm: 3 01:50:54,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=7, voteGranted=true] 01:50:54,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:54,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:50:54,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3] with higher term appears reachable, updating term. 01:50:54,941 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 8 in "RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3]" message is greater than follower's term 7 - updating term 01:50:54,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=null 01:50:54,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3] - currentTerm: 8, votedFor: null, lastIndex: 1, lastTerm: 3 01:50:54,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=8, voteGranted=true] 01:50:54,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:54,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:50:54,941 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3] with higher term appears reachable, updating term. 01:50:54,942 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 9 in "RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3]" message is greater than follower's term 8 - updating term 01:50:54,942 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=null 01:50:54,942 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3] - currentTerm: 9, votedFor: null, lastIndex: 1, lastTerm: 3 01:50:54,942 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,942 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=9, voteGranted=true] 01:50:54,942 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:54,942 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = null 01:50:54,942 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 01:50:54,942 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 01:50:54,942 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 01:50:54,942 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:54,942 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:54,944 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 1, nextIndex: 2 01:50:54,944 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:54,948 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - New leader address: akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,948 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,948 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 0 for term 9 01:50:54,948 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 01:50:54,984 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:50:54,985 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1555179585] 01:50:54,985 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 Queuing TransactionOperation 01:50:54,985 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 Readying 1 components for commit 01:50:54,985 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 Readying transaction for shard cars 01:50:54,985 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 Queuing TransactionOperation 01:50:54,985 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 canCommit 01:50:54,985 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:54,986 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1589003230, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@26c4face] 01:50:54,986 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1589003230, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@26c4face] 01:50:54,986 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1589003230)] for shard cars 01:50:54,986 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 01:50:54,986 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@46bc8c64 01:50:54,986 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 = [ 1 ] 01:50:54,986 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1589003230)] 01:50:54,987 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: handleReadyLocalTransaction for member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:50:54,987 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying local modifications for Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:50:54,987 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Validating transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:50:54,987 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 validated 01:50:54,987 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:54,987 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: canCommit for member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: success 01:50:54,987 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:50:54,987 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Preparing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:50:54,987 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 candidate ready 01:50:54,987 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: canCommit - candidate rootPath: / 01:50:54,987 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: canCommit - messages: [] 01:50:54,987 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 01:50:54,987 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 prepared 01:50:54,987 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:54,987 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Finishing commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:50:54,987 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Starting commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:50:54,987 PM [cluster-test-shard-dispatcher-26] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 01:50:54,987 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=2, term=9, payload=Simple{size=240}] 01:50:54,987 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=9, payload=Simple{size=240}] 01:50:54,988 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:54,988 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:50:54,988 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 2 is present for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,988 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=9, payload=Simple{size=240}]]] 01:50:54,988 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 submitted to persistence 01:50:54,988 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=2, term=9, payload=Simple{size=240}] 01:50:54,990 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=9, payload=Simple{size=240}]]] 01:50:54,990 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 01:50:54,990 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:50:54,990 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log Simple{size=240} 01:50:54,990 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=9, payload=Simple{size=240}] 01:50:54,990 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 2 01:50:54,990 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=9, success=true, followerId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=2, logLastTerm=9, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:54,991 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 2, nextIndex: 3 01:50:54,991 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:50:54,991 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 2 01:50:54,991 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 1 to 2 01:50:54,991 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 2 data Simple{size=240} 01:50:54,991 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Resuming commit of transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:50:54,991 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: commit - successfulFromPrevious: [] 01:50:54,991 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 01:50:54,991 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSingleShardTransactionsWithLeaderChanges_member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges%23-1589003230$b] 01:50:54,991 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 1 for term 9 01:50:54,991 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:54,991 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=2, term=9, payload=Simple{size=240}] 01:50:54,991 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=2 01:50:54,992 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 successfully completed direct commit 01:50:54,992 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 committed successfully 01:50:54,992 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 01:50:54,992 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 81.52 μs 01:50:54,992 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0 01:50:54,992 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1589003230)] for shard cars, trying to use DirectTransactionContextWrapper 01:50:54,992 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@46bc8c64 01:50:54,992 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0: Local transaction context created successfully, using DirectTransactionWrapper 01:50:54,992 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} 01:50:54,993 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Issuing gracefulStop to shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,993 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges - waiting on 1 shards 01:50:54,993 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onShutDown 01:50:54,993 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:54,993 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:50:54,994 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 01:50:54,994 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All shards shutdown - sending PoisonPill to self 01:50:54,994 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1589003230] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#1936383051] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#1936383051] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:54,994 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:50:54,994 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:54,995 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:50:54,995 PM [main] [WARN] ActorUtils - ShardManager for testSingleShardTransactionsWithLeaderChanges data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:50:54,995 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:54,996 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:50:54,996 PM [main] [WARN] ActorUtils - ShardManager for testSingleShardTransactionsWithLeaderChanges data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:50:54,996 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:54,997 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:54,998 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:54,998 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:55,2 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:55,2 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:55,3 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:55,3 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:55,15 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:50:55,16 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:50:55,16 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageSlicer - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-550: Closing 01:50:55,16 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 01:50:55,16 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:50:55,17 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:55,17 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:55,891 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is removing unreachable node [akka://cluster-test@127.0.0.1:2558] 01:50:55,893 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [-4849018227947970292] but current is [6027569023003710967]. Reason: Cluster member removed, previous status [Down] 01:50:56,18 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [6027569023003710967] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:56,34 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:56,34 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:50:56,36 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:56.037] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:56,40 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [9200127487165592706] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:50:56,44 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:50:56,44 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:50:56,46 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:50:56.048] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:56,70 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:50:56,77 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-425215940644961947] 01:50:56,77 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:50:56,79 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:50:56,79 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:50:56,79 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:56,79 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:56,96 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:50:56,96 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:50:56,96 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:50:56,99 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:56,108 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-7509161488635429466] 01:50:56,108 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:50:56,111 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:56,112 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:50:56,119 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:56,119 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:56,136 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:50:56,153 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [1405578497980769754] 01:50:56,153 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:50:56,160 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:50:56,160 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:50:56,165 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:50:56,165 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:50:56,170 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:56,172 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:56,173 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 01:50:56,183 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:50:56,185 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:56,197 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#1574763347] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:56,197 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 01:50:56,217 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:50:56,219 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:50:56,224 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 01:50:56,224 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:56,224 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:56,224 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:56,224 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: no initial generation override, starting from 0 01:50:56,224 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:50:56,224 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@75b99ec4 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@52eacf0d 01:50:56,224 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@56d54188 01:50:56,225 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1710165056224, None)) 01:50:56,227 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165056223,0,0)) 01:50:56,227 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@52eacf0d to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5dbd30ff 01:50:56,227 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3162d698 01:50:56,227 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} started 01:50:56,228 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:56,229 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:56,229 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:56,229 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:56,229 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:56,229 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-1-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927] 01:50:56,230 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:56,230 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithIsolatedLeader, persistent : true 01:50:56,231 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 01:50:56,232 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 01:50:56,233 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:56,233 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/shard-member-1:snapshot-read#1125309908] 01:50:56,233 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#-947294439 created and ready for shard:member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:56,233 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#-947294439] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:56,234 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/shard-member-1:snapshot-read#1125309908] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:56,236 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1597001838] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:56,237 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:56,237 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:56,237 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 01:50:56,237 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithIsolatedLeader is initialized 01:50:56,237 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:56,237 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 01:50:56,237 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 01:50:56,237 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 01:50:56,237 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from null to Follower 01:50:56,238 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 01:50:56,242 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 01:50:56,242 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:56,242 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: no initial generation override, starting from 0 01:50:56,242 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:50:56,242 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7a5590e4 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@232010be 01:50:56,243 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:56,243 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:56,245 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@271f9002 01:50:56,245 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1710165056245, None)) 01:50:56,245 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165056244,0,0)) 01:50:56,246 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@ff6ded5 01:50:56,246 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@232010be to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@53351a97 01:50:56,246 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} started 01:50:56,246 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:56,246 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:56,247 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:56,247 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:56,247 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-71494860] 01:50:56,247 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithIsolatedLeader, persistent : true 01:50:56,247 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 01:50:56,247 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-71494860] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:56,247 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#2093081586] 01:50:56,248 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#1594524745 created and ready for shard:member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:56,248 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#2093081586] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:56,248 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#1594524745] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:56,248 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:56,248 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:56,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 01:50:56,248 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithIsolatedLeader is initialized 01:50:56,248 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:56,248 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 01:50:56,248 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 01:50:56,248 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:56,248 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 01:50:56,248 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-2-shard-cars-testTransactionWithIsolatedLeader from null to Follower 01:50:56,296 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#1574763347] 01:50:56,297 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: findLocalShard : cars 01:50:56,297 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927] 01:50:56,547 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:50:56,547 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Received ElectionTimeout - switching to Candidate 01:50:56,547 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 01:50:56,547 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:56,547 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 1 01:50:56,547 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:50:56,547 PM [cluster-test-shard-dispatcher-15] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 01:50:56,547 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:56,547 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 01:50:56,548 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Follower to Candidate 01:50:56,548 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Follower to Candidate 01:50:56,777 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 01:50:56,777 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:56,777 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 2 01:50:56,866 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:57,67 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 01:50:57,67 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:57,67 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 3 01:50:57,97 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:50:57,97 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:50:57,98 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:57,98 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 01:50:57,98 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:57,98 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:57,98 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:57,98 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:57,98 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:57,105 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:57,105 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 01:50:57,105 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:57,105 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:57,106 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:57,106 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:57,176 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:50:57,327 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 01:50:57,327 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:57,327 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 4 01:50:57,327 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:57,329 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:50:57,330 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Cluster state: [] 01:50:57,330 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:50:57,330 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:50:57,330 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=null 01:50:57,330 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:50:57,330 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:57,330 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:50:57,331 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:50:57,331 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:50:57,331 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 01:50:57,331 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:57,332 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:50:57,332 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:50:57,332 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: became leader with frontend state for [] 01:50:57,332 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Candidate to Leader 01:50:57,332 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 01:50:57,332 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:57,332 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:57,332 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Candidate to Leader 01:50:57,332 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 01:50:57,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader: Closing 01:50:57,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithIsolatedLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:57,333 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 01:50:57,333 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:57,333 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:57,333 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 01:50:57,333 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done false 01:50:57,334 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: -1, nextIndex: 0 01:50:57,335 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:50:57,354 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionWithIsolatedLeader commit rateLimit adjusted to 80.58185946995822 01:50:57,354 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:50:57,354 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#1574763347] 01:50:57,354 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 Queuing TransactionOperation 01:50:57,354 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 01:50:57,354 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#1574763347] 01:50:57,354 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 Queuing TransactionOperation 01:50:57,355 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:50:57,355 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#1574763347] 01:50:57,355 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 Queuing TransactionOperation 01:50:57,355 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:57,355 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4b56d0b7] 01:50:57,355 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:57,355 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4b56d0b7] 01:50:57,356 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:50:57,356 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4b56d0b7] 01:50:57,356 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4b56d0b7] 01:50:57,356 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4b56d0b7] 01:50:57,356 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4b56d0b7] 01:50:57,356 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927)] for shard cars 01:50:57,356 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 01:50:57,356 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@56da148f 01:50:57,356 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927)] for shard cars 01:50:57,356 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@56da148f 01:50:57,356 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927)] for shard cars 01:50:57,356 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@56da148f 01:50:57,365 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:50:57,366 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:57,366 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 01:50:57,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onShutDown 01:50:57,366 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:57,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 01:50:57,366 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 01:50:57,366 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 01:50:57,366 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 01:50:57,367 PM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:50:57,367 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 Readying 1 components for commit 01:50:57,367 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 Readying transaction for shard cars 01:50:57,367 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 = [ 1 ] 01:50:57,367 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: shutting down 01:50:57,367 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:50:57,367 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927)] 01:50:57,367 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: handleReadyLocalTransaction for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:50:57,367 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, disabling transaction tracking 01:50:57,367 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1597001838] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1597001838] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:57,367 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1597001838] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1597001838] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:57,367 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#1038411595] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1038411595] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:57,367 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 01:50:57,367 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 01:50:57,367 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:50:57,367 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:50:57,367 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:57,367 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithIsolatedLeader 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}]]] 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying local modifications for Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 validated 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: canCommit for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: success 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Committing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Preparing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 candidate ready 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: canCommit - messages: [] 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 prepared 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Finishing commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Starting commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:50:57,368 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#1574763347] 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 170 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 01:50:57,368 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:57,368 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: findLocalShard : cars 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:57,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 submitted to persistence 01:50:57,369 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927] 01:50:57,369 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 01:50:57,369 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927] 01:50:57,369 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:57,375 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 01:50:57,375 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 01:50:57,425 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927] 01:50:57,447 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:57,476 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927] 01:50:57,527 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927] 01:50:57,567 PM [cluster-test-shard-dispatcher-15] [WARN] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: At least 1 followers need to be active, Switching member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 01:50:57,568 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 01:50:57,568 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:57,568 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:50:57,569 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 4 01:50:57,569 PM [cluster-test-shard-dispatcher-15] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader)_slicer-id-555: Closing 01:50:57,569 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:50:57,569 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Leader to IsolatedLeader 01:50:57,569 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 01:50:57,570 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:57,577 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927] 01:50:57,578 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 Readying 1 components for commit 01:50:57,578 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 Readying transaction for shard cars 01:50:57,578 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 = [ 1 ] 01:50:57,578 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927)] 01:50:57,578 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: handleReadyLocalTransaction for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 01:50:57,578 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionWithIsolatedLeader: Adding message org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction@7a69d082 to retry 01:50:57,578 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 canCommit 01:50:57,997 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionWithIsolatedLeader: Timer expired for message org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction@7a69d082 01:50:57,997 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] LocalThreePhaseCommitCohort - Failed to prepare transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 on backend org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Could not process ready local transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0. Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.ShardTransactionMessageRetrySupport$MessageInfo.timedOut(ShardTransactionMessageRetrySupport.java:102) at org.opendaylight.controller.cluster.datastore.ShardTransactionMessageRetrySupport.onTimerMessage(ShardTransactionMessageRetrySupport.java:72) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:410) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:50:57,998 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithIsolatedLeader 01:50:57,998 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 Readying 1 components for commit 01:50:57,998 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 Readying transaction for shard cars 01:50:57,998 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 = [ 1 ] 01:50:57,999 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-964173927)] 01:50:57,999 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:57,999 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: handleReadyLocalTransaction for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:50:57,999 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:57,999 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionWithIsolatedLeader: Adding message org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction@254a0f2b to retry 01:50:57,999 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 01:50:58,1 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:50:58,4 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 01:50:58,6 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 01:50:58,7 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: recovered identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:50:58,8 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 01:50:58,8 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:58,8 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} 01:50:58,8 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6cfd707b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@d37f318 01:50:58,8 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@178e1666 01:50:58,8 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:50:58,8 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 01:50:58,8 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:50:58,8 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:58,8 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:50:58,8 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:58,9 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1710165058008, None)) 01:50:58,12 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165058007,0,0)) 01:50:58,12 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@d37f318 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1b21a24d 01:50:58,12 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@23e74e63 01:50:58,12 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} started 01:50:58,13 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 canCommit 01:50:58,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:50:58,13 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:58,14 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:58,14 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:58,14 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#103870012] 01:50:58,14 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:58,14 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:58,14 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-2-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#103870012] 01:50:58,14 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 01:50:58,14 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:58,14 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-3-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#103870012] 01:50:58,14 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithIsolatedLeader, persistent : true 01:50:58,15 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#1753671629] 01:50:58,15 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 01:50:58,17 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:50:58,17 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=null 01:50:58,17 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:50:58,17 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:58,17 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#-1621607261 created and ready for shard:member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:58,17 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:50:58,17 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:50:58,17 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 01:50:58,17 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithIsolatedLeader is initialized 01:50:58,17 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:50:58,17 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 01:50:58,17 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 01:50:58,17 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:58,17 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 01:50:58,18 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-2-shard-cars-testTransactionWithIsolatedLeader from null to Follower 01:50:58,46 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:58,47 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}]]] 01:50:58,60 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}]]] 01:50:58,60 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 2 01:50:58,60 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:50:58,60 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60} 01:50:58,60 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 01:50:58,60 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log Simple{size=170} 01:50:58,60 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 01:50:58,60 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 2 01:50:58,60 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:58,60 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader: Closing 01:50:58,60 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithIsolatedLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:58,61 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 01:50:58,61 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:58,61 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:58,61 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 01:50:58,61 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 01:50:58,61 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 01:50:58,61 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done false 01:50:58,62 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 1, nextIndex: 2 01:50:58,62 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 01:50:58,62 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Setting last applied to 0 01:50:58,62 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from -1 to 0 01:50:58,62 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60} 01:50:58,62 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:50:58,62 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionWithIsolatedLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} does not match any client, pre-disabling client. 01:50:58,62 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, nanosAgo=694865773} is already disabled 01:50:58,62 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Setting last applied to 1 01:50:58,62 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 0 to 1 01:50:58,62 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 1 data Simple{size=170} 01:50:58,62 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Resuming commit of transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:50:58,62 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:50:58,62 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:50:58,62 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionWithIsolatedLeader_member-1-shard-cars-testTransactionWithIsolatedLeader%23-964173927$a] 01:50:58,62 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionWithIsolatedLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#2110024767] with name member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher 01:50:58,63 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 0 for term 4 01:50:58,63 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:58,63 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - IsolatedLeader member-1-shard-cars-testTransactionWithIsolatedLeader switching from IsolatedLeader to Leader 01:50:58,63 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 01:50:58,63 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:50:58,63 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 4 01:50:58,63 PM [cluster-test-shard-dispatcher-15] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader)_slicer-id-556: Closing 01:50:58,63 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:50:58,63 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionWithIsolatedLeader: Retrying message org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction@254a0f2b 01:50:58,63 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=1 01:50:58,63 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: handleReadyLocalTransaction for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:50:58,63 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying local modifications for Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:50:58,63 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:50:58,63 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 validated 01:50:58,63 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:50:58,63 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: canCommit for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: success 01:50:58,63 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Committing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:50:58,63 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Preparing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:50:58,64 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 candidate ready 01:50:58,64 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: canCommit - candidate rootPath: / 01:50:58,64 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 01:50:58,64 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: canCommit - messages: [] 01:50:58,64 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 01:50:58,64 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 prepared 01:50:58,64 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:50:58,64 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Finishing commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:50:58,64 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Starting commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:50:58,64 PM [cluster-test-shard-dispatcher-15] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 68 01:50:58,64 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 01:50:58,64 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 01:50:58,64 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:50:58,64 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 successfully completed direct commit 01:50:58,64 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 committed successfully 01:50:58,64 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#2110024767] 01:50:58,64 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from IsolatedLeader to Leader 01:50:58,64 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 01:50:58,65 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#2110024767] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:50:58,65 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 99.55 μs 01:50:58,65 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 canCommit 01:50:58,65 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:50:58,65 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:58,65 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}]]] 01:50:58,65 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 submitted to persistence 01:50:58,65 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from IsolatedLeader to Leader 01:50:58,65 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 01:50:58,66 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 01:50:58,67 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 1 01:50:58,67 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 01:50:58,67 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 0 01:50:58,67 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from -1 to 0 01:50:58,67 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60} 01:50:58,67 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:50:58,67 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionWithIsolatedLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} does not match any client, pre-disabling client. 01:50:58,67 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 1 01:50:58,67 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 0 to 1 01:50:58,67 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 1 data Simple{size=170} 01:50:58,67 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying foreign transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:50:58,68 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionWithIsolatedLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#1301817275] with name member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher 01:50:58,68 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 0 for term 4 01:50:58,68 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=1 01:50:58,68 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done true 01:50:58,68 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#1301817275] 01:50:58,68 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 01:50:58,68 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 113.1 μs 01:50:58,68 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}]]] 01:50:58,68 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 1 01:50:58,68 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:50:58,68 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log Simple{size=68} 01:50:58,68 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 01:50:58,68 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 2 01:50:58,68 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:50:58,69 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 01:50:58,69 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:58,69 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:50:58,69 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 2, nextIndex: 3 01:50:58,69 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:50:58,69 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 2 01:50:58,69 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 1 to 2 01:50:58,69 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 2 data Simple{size=68} 01:50:58,69 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Resuming commit of transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:50:58,69 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: commit - successfulFromPrevious: [] 01:50:58,69 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 01:50:58,70 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionWithIsolatedLeader_member-1-shard-cars-testTransactionWithIsolatedLeader%23-964173927$b] 01:50:58,70 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 committed successfully 01:50:58,70 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 01:50:58,70 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 successfully completed direct commit 01:50:58,70 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 1 for term 4 01:50:58,70 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 16.11 μs 01:50:58,70 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:58,70 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=2 01:50:58,70 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:50:58,71 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-1-shard-cars-testTransactionWithIsolatedLeader 01:50:58,71 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 01:50:58,71 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onShutDown 01:50:58,71 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Initiating leader transfer 01:50:58,71 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2c7c89f1 01:50:58,71 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2c7c89f1 01:50:58,71 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Attempting to transfer leadership 01:50:58,71 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderId=null, leaderPayloadVersion=13] 01:50:58,71 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:58,71 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:58,71 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:58,71 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 2 01:50:58,71 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:50:58,71 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 2 01:50:58,72 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 1 to 2 01:50:58,72 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 2 data Simple{size=68} 01:50:58,72 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying foreign transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:50:58,72 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 1 for term 4 01:50:58,72 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=2 01:50:58,72 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader] 01:50:58,72 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 01:50:58,72 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 15.86 μs 01:50:58,72 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=null, leaderPayloadVersion=13] 01:50:58,72 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:50:58,72 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:50:58,72 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionWithIsolatedLeader, matchIndex: 2, lastIndex: 2, isVoting: true 01:50:58,72 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Follower's log matches - sending ElectionTimeout 01:50:58,72 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:50:58,73 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Leader transfer complete 01:50:58,75 PM [cluster-test-shard-dispatcher-24] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:50:58,117 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:51:00,87 PM [cluster-test-shard-dispatcher-29] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testTransactionWithIsolatedLeader: Successfully transferred leadership to member-1-shard-cars-testTransactionWithIsolatedLeader in 2.016 s 01:51:00,87 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: leader transfer succeeded - sending PoisonPill 01:51:00,87 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithIsolatedLeader 01:51:00,87 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader)_slicer-id-559: Closing 01:51:00,88 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 01:51:00,88 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 01:51:00,88 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 01:51:00,88 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:51:00,88 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:51:00,89 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:51:00,90 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#1574763347] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#1574763347] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:00,90 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:51:00,90 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:00,90 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:00,99 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:00,99 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:00,99 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:00,99 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:00,103 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:51:00,110 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:51:00,111 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithIsolatedLeader 01:51:00,111 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 01:51:00,111 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 01:51:00,112 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 01:51:00,112 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:00,112 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:01,106 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-425215940644961947] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:01,126 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-7509161488635429466] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:01,129 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:01,129 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:01,133 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:01.135] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:01,148 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:01,148 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:01,151 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:01.152] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:01,166 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:51:01,174 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-4081151910946900126] 01:51:01,174 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:51:01,176 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:51:01,176 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:51:01,176 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:01,176 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:01,190 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 closeTransaction called 01:51:01,190 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_33#1154807177)] 01:51:01,198 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:51:01,198 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:51:01,200 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:51:01,203 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:51:01,209 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-4813675199689500839] 01:51:01,209 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:51:01,215 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:01,216 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:51:01,216 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:01,216 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:01,274 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:51:01,283 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-5009188365175470829] 01:51:01,283 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:51:01,284 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:51:01,285 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:01,286 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:01,286 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:51:01,286 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:01,286 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:01,297 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:01,299 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:01,301 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithMultipleShards 01:51:01,308 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: no initial generation override, starting from 0 01:51:01,308 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:51:01,308 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@60a26ac8 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5180cd08 01:51:01,308 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@795b56a2 01:51:01,308 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testWriteTransactionWithMultipleShards, 0, 1710165061308, None)) 01:51:01,309 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165061307,0,0)) 01:51:01,309 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-1085859189] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,309 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5180cd08 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2ce146f5 01:51:01,309 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} started 01:51:01,310 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithMultipleShards 01:51:01,310 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:01,312 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:01,313 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithMultipleShards 01:51:01,316 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithMultipleShards 01:51:01,316 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-404391560] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,320 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:51:01,320 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithMultipleShards 01:51:01,320 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,320 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,320 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4efcfb00 01:51:01,320 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:01,320 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,320 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,320 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:01,321 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,322 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1850801333] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,322 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testWriteTransactionWithMultipleShards, persistent : true 01:51:01,322 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,322 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,322 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards on shard member-1-shard-cars-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1850801333] 01:51:01,322 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,322 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,322 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,322 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards on shard member-1-shard-people-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1009267373] 01:51:01,322 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-1-shard-people-testWriteTransactionWithMultipleShards, persistent : true 01:51:01,323 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1009267373] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,323 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testWriteTransactionWithMultipleShards with journal batch size 1 01:51:01,324 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:01,325 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: no initial generation override, starting from 0 01:51:01,325 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:51:01,325 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithMultipleShards 01:51:01,325 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2e5e500a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@612ef3ef 01:51:01,325 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,325 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5f1b0583 01:51:01,325 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:01,325 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:01,325 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,325 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,325 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithMultipleShards, 0, 1710165061325, None)) 01:51:01,325 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165061324,0,0)) 01:51:01,326 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@612ef3ef to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7d1f2745 01:51:01,326 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} started 01:51:01,326 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@11132999 01:51:01,326 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:01,326 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,326 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,326 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,326 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards on shard member-2-shard-cars-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards#-696872600] 01:51:01,326 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:01,327 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:01,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards on shard member-2-shard-people-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards#1729715150] 01:51:01,327 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-people-testWriteTransactionWithMultipleShards, persistent : true 01:51:01,327 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards#1729715150] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,327 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards#-696872600] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,327 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithMultipleShards, persistent : true 01:51:01,327 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-notifier#8311070 created and ready for shard:member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:01,327 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#677208491] 01:51:01,327 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-notifier#8311070] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,327 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-notifier#-1566137324 created and ready for shard:member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,327 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#273064479] 01:51:01,327 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-people-testWriteTransactionWithMultipleShards with journal batch size 1 01:51:01,328 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-43852493] 01:51:01,328 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-notifier#620649536 created and ready for shard:member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,328 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-notifier#620649536] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,328 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-411634297] 01:51:01,328 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:01,328 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:01,328 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-43852493] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,328 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-411634297] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,328 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:01,328 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 01:51:01,328 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [people] 01:51:01,328 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,328 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-testWriteTransactionWithMultipleShards is initialized 01:51:01,328 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithMultipleShards , received role change from null to Follower 01:51:01,328 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 01:51:01,328 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-2-shard-people-testWriteTransactionWithMultipleShards from null to Follower 01:51:01,329 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-notifier#1536288558 created and ready for shard:member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,329 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-notifier#1536288558] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,329 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#677208491] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,329 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-notifier#-1566137324] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,329 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#273064479] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:01,329 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:01,329 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:01,329 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-1-shard-people-testWriteTransactionWithMultipleShards with journal batch size 1 01:51:01,329 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithMultipleShards with journal batch size 1 01:51:01,329 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [cars] 01:51:01,329 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-1-shard-cars-testWriteTransactionWithMultipleShards is initialized 01:51:01,330 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:01,330 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 01:51:01,330 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from null to Follower 01:51:01,330 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 01:51:01,331 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from null to Follower 01:51:01,336 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:01,336 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:01,336 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [cars] 01:51:01,336 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithMultipleShards is initialized 01:51:01,336 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:01,336 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 01:51:01,336 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-1-shard-cars-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,337 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithMultipleShards , received role change from null to Follower 01:51:01,337 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 01:51:01,337 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-2-shard-cars-testWriteTransactionWithMultipleShards from null to Follower 01:51:01,337 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:01,337 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:01,337 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [people] 01:51:01,337 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-1-shard-people-testWriteTransactionWithMultipleShards is initialized 01:51:01,337 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:01,337 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 01:51:01,337 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from null to Follower 01:51:01,338 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 01:51:01,338 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from null to Follower 01:51:01,376 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-1085859189] 01:51:01,376 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: findLocalShard : cars 01:51:01,376 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1850801333] 01:51:01,585 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:01,586 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:51:01,586 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testWriteTransactionWithMultipleShards] 01:51:01,586 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,586 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 01:51:01,586 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:01,586 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Closing 01:51:01,586 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:01,586 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 01:51:01,586 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from Follower to Candidate 01:51:01,586 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from Follower to Candidate 01:51:01,651 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:01,651 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:51:01,652 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testWriteTransactionWithMultipleShards] 01:51:01,652 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,652 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 01:51:01,652 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:01,652 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Closing 01:51:01,652 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:01,652 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 01:51:01,652 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from Follower to Candidate 01:51:01,652 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from Follower to Candidate 01:51:01,811 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:51:01,811 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:01,811 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 01:51:01,911 PM [cluster-test-shard-dispatcher-23] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:01,921 PM [cluster-test-shard-dispatcher-23] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:01,961 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:51:01,961 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:01,961 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 01:51:02,111 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:51:02,111 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,111 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 01:51:02,193 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:51:02,193 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:51:02,194 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:02,194 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,194 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,194 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards 01:51:02,194 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:02,194 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:02,194 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,194 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,194 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:02,194 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:02,194 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-2-shard-people-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:02,194 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,202 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:02,202 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,202 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,202 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards 01:51:02,202 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:02,202 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:02,202 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,202 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,202 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:02,202 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:02,221 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:51:02,221 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:02,221 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 01:51:02,221 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:02,222 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:02,222 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Cluster state: [] 01:51:02,222 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:02,222 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:02,222 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=null 01:51:02,223 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:02,223 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:02,223 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 01:51:02,224 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 01:51:02,224 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 01:51:02,224 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testWriteTransactionWithMultipleShards] 01:51:02,224 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:02,224 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 01:51:02,224 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:51:02,224 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: became leader with frontend state for [] 01:51:02,224 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from Candidate to Leader 01:51:02,224 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:51:02,224 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:02,224 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:02,224 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from Candidate to Leader 01:51:02,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithMultipleShards: Closing 01:51:02,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:02,225 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:51:02,225 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:02,225 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:02,225 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testWriteTransactionWithMultipleShards status sync done false 01:51:02,226 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:51:02,226 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:02,298 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:51:02,341 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:51:02,341 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,341 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 01:51:02,341 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,343 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:02,343 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Cluster state: [] 01:51:02,343 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:02,343 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:02,343 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 01:51:02,343 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:02,343 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,343 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testWriteTransactionWithMultipleShards status sync done true 01:51:02,343 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:02,345 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:02,345 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:02,345 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testWriteTransactionWithMultipleShards] 01:51:02,345 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:02,345 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:02,345 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:51:02,345 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: became leader with frontend state for [] 01:51:02,345 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from Candidate to Leader 01:51:02,345 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:51:02,345 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:02,345 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:02,345 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from Candidate to Leader 01:51:02,345 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All Shards are ready - data store testWriteTransactionWithMultipleShards is ready 01:51:02,346 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithMultipleShards: Closing 01:51:02,346 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,346 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithMultipleShards status sync done false 01:51:02,346 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:51:02,346 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:02,346 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:02,346 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All Shards are ready - data store testWriteTransactionWithMultipleShards is ready 01:51:02,351 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:51:02,351 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:02,437 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-1085859189] 01:51:02,437 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: findLocalShard : people 01:51:02,437 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1009267373] 01:51:02,438 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testWriteTransactionWithMultipleShards commit rateLimit adjusted to 80.58185946995822 01:51:02,438 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,439 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-404391560] 01:51:02,439 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 01:51:02,439 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-404391560] 01:51:02,439 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 01:51:02,439 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying 2 components for commit 01:51:02,439 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying transaction for shard cars 01:51:02,439 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 01:51:02,439 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying transaction for shard people 01:51:02,439 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 01:51:02,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:02,439 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 canCommit 01:51:02,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards, primaryVersion=13] 01:51:02,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:51:02,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards, primaryVersion=13] 01:51:02,439 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards, primaryVersion=13] 01:51:02,439 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards, primaryVersion=13] 01:51:02,440 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] for shard people 01:51:02,440 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] found - creating WRITE_ONLY transaction context 01:51:02,440 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] for shard cars 01:51:02,440 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] found - creating WRITE_ONLY transaction context 01:51:02,440 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 01:51:02,440 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 = [ 1 ] 01:51:02,440 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction called 01:51:02,440 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:51:02,440 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 sending 1 batched modifications, ready: true 01:51:02,440 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 = [ 1 ] 01:51:02,440 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] 01:51:02,440 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction called 01:51:02,440 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 sending 1 batched modifications, ready: true 01:51:02,440 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] 01:51:02,443 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, disabling transaction tracking 01:51:02,444 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 01:51:02,444 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 01:51:02,444 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, disabling transaction tracking 01:51:02,444 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:51:02,444 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:02,444 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:02,444 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 01:51:02,444 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 01:51:02,444 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}]]] 01:51:02,444 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:51:02,444 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,444 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:02,444 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,444 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}]]] 01:51:02,444 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Readying Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 of 1 operations, client version 13 01:51:02,444 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,444 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 ready 01:51:02,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Readying Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 of 1 operations, client version 13 01:51:02,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 ready 01:51:02,445 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 01:51:02,445 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 01:51:02,446 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}]]] 01:51:02,446 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:51:02,446 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:02,446 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 01:51:02,446 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 01:51:02,446 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction 01:51:02,446 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Log size is now 1 01:51:02,446 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:02,446 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1009267373)] resolved 01:51:02,446 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:51:02,446 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction 01:51:02,446 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}]]] 01:51:02,446 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1850801333)] resolved 01:51:02,446 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:51:02,446 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: successfully resolved all cohort actors 01:51:02,446 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:02,446 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 01:51:02,446 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 01:51:02,446 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 finishCanCommit 01:51:02,447 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Log size is now 1 01:51:02,447 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:02,447 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1850801333)] 01:51:02,447 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1850801333)] 01:51:02,447 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 01:51:02,447 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 01:51:02,447 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 01:51:02,447 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithMultipleShards status sync done true 01:51:02,448 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:51:02,449 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:51:02,449 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 01:51:02,449 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:51:02,449 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 01:51:02,449 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:51:02,449 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:51:02,449 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:51:02,449 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-1-shard-people-testWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:51:02,449 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 01:51:02,449 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:51:02,449 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, nanosAgo=5250979} is already disabled 01:51:02,449 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 01:51:02,449 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:51:02,449 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-1-shard-cars-testWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:51:02,449 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, nanosAgo=5055944} is already disabled 01:51:02,449 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:02,449 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:02,449 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:51:02,449 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:51:02,449 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Can committing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,449 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,449 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 validated 01:51:02,449 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:02,449 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: canCommit for member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: success 01:51:02,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Commit index set to 0 01:51:02,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:51:02,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 01:51:02,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:51:02,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 01:51:02,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:51:02,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-people-testWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:51:02,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:51:02,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Commit index set to 0 01:51:02,450 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:51:02,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:51:02,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 01:51:02,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:51:02,450 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1009267373)] 01:51:02,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 01:51:02,450 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1009267373)] 01:51:02,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:51:02,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 01:51:02,450 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:51:02,457 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Can committing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,457 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,457 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 validated 01:51:02,457 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:02,457 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: canCommit for member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: success 01:51:02,458 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:51:02,458 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit returning result: true 01:51:02,458 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 commit 01:51:02,458 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 finish commit 01:51:02,458 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1850801333)] 01:51:02,458 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1850801333)] 01:51:02,458 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1009267373)] 01:51:02,458 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1009267373)] 01:51:02,459 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Committing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,459 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,459 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Committing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,459 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 candidate ready 01:51:02,459 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,459 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:51:02,459 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 candidate ready 01:51:02,459 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:51:02,459 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - messages: [] 01:51:02,459 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:51:02,459 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 prepared 01:51:02,459 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:02,459 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Finishing commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,459 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - messages: [] 01:51:02,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:51:02,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 prepared 01:51:02,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:02,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Finishing commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,460 PM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 175 01:51:02,460 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 01:51:02,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 179 01:51:02,460 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 01:51:02,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}] 01:51:02,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}] 01:51:02,460 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:02,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:02,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:51:02,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:02,460 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:51:02,460 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}]]] 01:51:02,460 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 submitted to persistence 01:51:02,460 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}]]] 01:51:02,460 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 submitted to persistence 01:51:02,461 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}] 01:51:02,461 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 01:51:02,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}]]] 01:51:02,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:51:02,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:51:02,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=179} 01:51:02,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}] 01:51:02,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Log size is now 2 01:51:02,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:02,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}]]] 01:51:02,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:51:02,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:51:02,462 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=175} 01:51:02,462 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 01:51:02,462 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Log size is now 2 01:51:02,462 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:02,463 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}] 01:51:02,463 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 01:51:02,464 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:51:02,464 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:51:02,464 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 01:51:02,464 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:51:02,464 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=179} 01:51:02,464 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:51:02,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:51:02,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testWriteTransactionWithMultipleShards_member-1-shard-people-testWriteTransactionWithMultipleShards%23-1009267373$f] 01:51:02,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-1037297860] with name member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher 01:51:02,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 3 01:51:02,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:02,465 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-1037297860] 01:51:02,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:51:02,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:51:02,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:51:02,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 01:51:02,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:51:02,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=175} 01:51:02,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,465 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:51:02,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:51:02,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:51:02,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testWriteTransactionWithMultipleShards_member-1-shard-cars-testWriteTransactionWithMultipleShards%23-1850801333$e] 01:51:02,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-1636553505] with name member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher 01:51:02,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:51:02,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:02,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:51:02,465 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher: 126.2 μs 01:51:02,466 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-1037297860] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:02,466 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-1636553505] 01:51:02,466 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:51:02,466 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher: 57.89 μs 01:51:02,466 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-1636553505] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:02,467 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit succeeded 01:51:02,467 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Commit index set to 1 01:51:02,467 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:51:02,467 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 01:51:02,467 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:51:02,467 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=179} 01:51:02,467 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:51:02,467 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,467 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] for shard cars, trying to use DirectTransactionContextWrapper 01:51:02,467 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:51:02,467 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] found - trying create transaction 01:51:02,467 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] 01:51:02,467 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Queuing TransactionOperation 01:51:02,467 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#355462587] with name member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher 01:51:02,468 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 3 01:51:02,468 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#355462587] 01:51:02,468 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:51:02,468 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:51:02,468 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Commit index set to 1 01:51:02,468 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:51:02,468 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 01:51:02,468 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:51:02,468 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=175} 01:51:02,468 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#355462587] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:02,468 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher: 109.8 μs 01:51:02,468 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:02,468 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-855620453] with name member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher 01:51:02,468 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-855620453] 01:51:02,468 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:51:02,468 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:51:02,469 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:51:02,469 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-855620453] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:02,469 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher: 46.21 μs 01:51:02,469 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:51:02,474 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_34#1721284947] 01:51:02,474 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_34#1721284947, transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0, version=13] 01:51:02,474 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_34#1721284947] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:02,474 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 01:51:02,474 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_34#1721284947)] 01:51:02,476 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 ReadData operation succeeded 01:51:02,476 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] for shard people, trying to use DirectTransactionContextWrapper 01:51:02,476 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 01:51:02,476 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] found - trying create transaction 01:51:02,476 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] 01:51:02,477 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Queuing TransactionOperation 01:51:02,477 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:51:02,477 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_35#-199199217] 01:51:02,478 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_35#-199199217, transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0, version=13] 01:51:02,478 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 01:51:02,478 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_35#-199199217)] 01:51:02,479 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 ReadData operation succeeded 01:51:02,479 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithMultipleShards - waiting on 2 shards 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onShutDown 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Initiating leader transfer 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@362b54f7 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@362b54f7 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onShutDown 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Initiating leader transfer 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7c584e00 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7c584e00 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:02,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:02,481 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards] 01:51:02,481 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards] 01:51:02,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:51:02,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:51:02,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:02,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Leader transfer complete 01:51:02,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:51:02,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:51:02,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:02,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Leader transfer complete 01:51:02,487 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:51:02,487 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:02,487 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:02,487 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:51:02,487 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:02,487 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:02,705 PM [cluster-test-shard-dispatcher-22] [WARN] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 221, lastApplied : 1, commitIndex : 1 01:51:02,705 PM [cluster-test-shard-dispatcher-32] [WARN] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 221, lastApplied : 1, commitIndex : 1 01:51:02,891 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 01:51:02,891 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1850801333]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:51:02,891 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-1-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,891 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader)_slicer-id-567: Closing 01:51:02,891 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-1-shard-people-testWriteTransactionWithMultipleShards 01:51:02,891 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader)_slicer-id-566: Closing 01:51:02,891 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:51:02,891 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:51:02,892 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithMultipleShards 01:51:02,892 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 01:51:02,892 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:51:02,893 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: shutting down 01:51:02,893 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 01:51:02,893 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 01:51:02,894 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:02,894 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:02,902 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:02,902 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:02,902 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:02,902 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:02,905 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:51:02,914 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithMultipleShards 01:51:02,914 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Closing 01:51:02,914 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:51:02,914 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-2-shard-people-testWriteTransactionWithMultipleShards 01:51:02,914 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Closing 01:51:02,914 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:51:02,914 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithMultipleShards 01:51:02,916 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 01:51:02,916 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:02,916 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:03,907 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-4081151910946900126] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:03,934 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:03,934 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:03,936 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:03.938] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:03,939 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-4813675199689500839] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:03,943 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:03,943 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:03,945 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:03.946] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:03,954 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:03,960 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3512659926226480510] 01:51:03,960 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:51:03,962 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:51:03,962 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:51:03,963 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:03,963 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:03,972 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:51:03,972 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:51:03,972 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:51:03,974 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:03,978 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [7796815374138719843] 01:51:03,978 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:51:03,980 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:03,980 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:51:03,986 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:03,986 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:04,10 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:04,23 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-541034110159990477] 01:51:04,23 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:51:04,23 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:04,23 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:51:04,23 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:04,23 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:04,30 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:51:04,33 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:04,33 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:04,35 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:04,37 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithMultipleShards 01:51:04,44 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#614344334] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:04,45 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithMultipleShards 01:51:04,46 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: no initial generation override, starting from 0 01:51:04,46 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:51:04,46 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2ad01cac to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@502533bd 01:51:04,47 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithMultipleShards 01:51:04,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-1-shard-cars-testTransactionChainWithMultipleShards 01:51:04,47 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@ce69d97 01:51:04,47 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionChainWithMultipleShards, 0, 1710165064046, None)) 01:51:04,47 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165064045,0,0)) 01:51:04,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-1-shard-people-testTransactionChainWithMultipleShards 01:51:04,47 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:04,47 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:51:04,47 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:51:04,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@39826a64 01:51:04,47 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@502533bd to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@24489754 01:51:04,47 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} started 01:51:04,48 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:04,48 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionChainWithMultipleShards 01:51:04,48 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:04,48 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:51:04,48 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:51:04,48 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-cars-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards on shard member-1-shard-cars-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-393591239] 01:51:04,48 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testTransactionChainWithMultipleShards 01:51:04,48 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionChainWithMultipleShards, persistent : true 01:51:04,49 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionChainWithMultipleShards with journal batch size 1 01:51:04,49 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-people-testTransactionChainWithMultipleShards, persistent : true 01:51:04,49 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-people-testTransactionChainWithMultipleShards with journal batch size 1 01:51:04,50 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-393591239] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:04,50 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#2003613132] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:04,50 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-notifier#824122925 created and ready for shard:member-1-shard-cars-testTransactionChainWithMultipleShards 01:51:04,50 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-notifier#824122925] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:04,50 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#672833689] 01:51:04,50 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:04,50 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-notifier#-1371476276 created and ready for shard:member-1-shard-people-testTransactionChainWithMultipleShards 01:51:04,50 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#1434989986] 01:51:04,50 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#672833689] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:04,50 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-notifier#-1371476276] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:04,50 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#1434989986] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:04,52 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithMultipleShards 01:51:04,52 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:51:04,52 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:51:04,52 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-people-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards on shard member-1-shard-people-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#2003613132] 01:51:04,53 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:04,53 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:04,53 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:04,53 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:04,53 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:04,53 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 01:51:04,53 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [people] 01:51:04,53 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-1-shard-people-testTransactionChainWithMultipleShards is initialized 01:51:04,53 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [cars] 01:51:04,53 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionChainWithMultipleShards is initialized 01:51:04,54 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from null to Follower 01:51:04,54 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 01:51:04,55 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from null to Follower 01:51:04,56 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithMultipleShards 01:51:04,56 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:04,56 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionChainWithMultipleShards is no longer the leader 01:51:04,57 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#285640401] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:04,57 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from null to Follower 01:51:04,57 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 01:51:04,57 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from null to Follower 01:51:04,58 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: no initial generation override, starting from 0 01:51:04,58 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithMultipleShards 01:51:04,58 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:51:04,58 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-2-shard-cars-testTransactionChainWithMultipleShards 01:51:04,58 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6b6a79dc to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3e7f6ad 01:51:04,58 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@627df187 01:51:04,58 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-2-shard-people-testTransactionChainWithMultipleShards 01:51:04,59 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionChainWithMultipleShards, 0, 1710165064058, None)) 01:51:04,59 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:04,59 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:51:04,59 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:51:04,59 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165064057,0,0)) 01:51:04,59 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@13dd88d8 01:51:04,59 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3e7f6ad to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4d9c949f 01:51:04,59 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} started 01:51:04,59 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:04,59 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionChainWithMultipleShards 01:51:04,60 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:51:04,60 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:51:04,60 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-cars-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards on shard member-2-shard-cars-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards#-1426919242] 01:51:04,60 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testTransactionChainWithMultipleShards 01:51:04,60 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionChainWithMultipleShards, persistent : true 01:51:04,60 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards#-1426919242] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:04,61 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionChainWithMultipleShards with journal batch size 1 01:51:04,61 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-2-shard-people-testTransactionChainWithMultipleShards, persistent : true 01:51:04,61 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:51:04,61 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:51:04,61 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-people-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards on shard member-2-shard-people-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards#1709767667] 01:51:04,61 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-2-shard-people-testTransactionChainWithMultipleShards with journal batch size 1 01:51:04,65 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-notifier#1695629780 created and ready for shard:member-2-shard-cars-testTransactionChainWithMultipleShards 01:51:04,65 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-1487108205] 01:51:04,65 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-notifier#1695629780] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:04,65 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-1919944772] 01:51:04,66 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:04,66 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:04,66 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:04,66 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:04,66 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards#1709767667] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:04,66 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-1487108205] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:04,66 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-1919944772] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:04,66 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:04,66 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:04,66 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 01:51:04,66 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionChainWithMultipleShards is no longer the leader 01:51:04,66 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithMultipleShards , received role change from null to Follower 01:51:04,66 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-1-shard-people-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:51:04,66 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:51:04,67 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [people] 01:51:04,67 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-people-testTransactionChainWithMultipleShards is initialized 01:51:04,67 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [cars] 01:51:04,67 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionChainWithMultipleShards is initialized 01:51:04,67 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 01:51:04,67 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-2-shard-cars-testTransactionChainWithMultipleShards from null to Follower 01:51:04,67 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-notifier#1018995344 created and ready for shard:member-2-shard-people-testTransactionChainWithMultipleShards 01:51:04,67 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionChainWithMultipleShards , received role change from null to Follower 01:51:04,67 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 01:51:04,67 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-notifier#1018995344] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:04,67 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-2-shard-people-testTransactionChainWithMultipleShards from null to Follower 01:51:04,73 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:51:04,79 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:04,109 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#614344334] 01:51:04,110 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: findLocalShard : cars 01:51:04,110 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-393591239] 01:51:04,352 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:04,352 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:51:04,352 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionChainWithMultipleShards] 01:51:04,352 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 01:51:04,352 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 1 01:51:04,352 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:04,352 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Closing 01:51:04,352 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:04,352 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 01:51:04,352 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from Follower to Candidate 01:51:04,353 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from Follower to Candidate 01:51:04,362 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:04,362 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:51:04,362 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testTransactionChainWithMultipleShards] 01:51:04,362 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 01:51:04,362 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 1 01:51:04,362 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:04,362 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Closing 01:51:04,362 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:04,362 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionChainWithMultipleShards is no longer the leader 01:51:04,362 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from Follower to Candidate 01:51:04,362 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from Follower to Candidate 01:51:04,603 PM [cluster-test-shard-dispatcher-25] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:04,641 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:51:04,641 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 01:51:04,641 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 2 01:51:04,653 PM [cluster-test-shard-dispatcher-25] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:04,671 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:51:04,671 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 01:51:04,671 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 2 01:51:04,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:51:04,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 01:51:04,902 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 3 01:51:04,941 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:51:04,941 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 01:51:04,941 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 3 01:51:04,983 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:51:04,983 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:51:04,984 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:04,984 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards 01:51:04,984 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:51:04,985 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards 01:51:04,985 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:51:04,985 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:04,985 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:51:04,985 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:51:04,985 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:51:04,985 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:51:04,985 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:51:04,985 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-people-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:51:04,992 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:04,992 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards 01:51:04,992 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:51:04,992 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards 01:51:04,992 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:51:04,992 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:04,992 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:51:04,992 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 01:51:04,992 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:51:04,992 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 01:51:05,41 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:51:05,161 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:51:05,162 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 01:51:05,162 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 4 01:51:05,162 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testTransactionChainWithMultipleShards 01:51:05,163 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:05,163 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Cluster state: [] 01:51:05,163 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:05,163 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:05,163 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=null 01:51:05,163 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:05,163 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 01:51:05,164 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:05,165 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:05,165 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:05,165 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testTransactionChainWithMultipleShards] 01:51:05,166 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:05,166 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:05,166 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:51:05,166 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: became leader with frontend state for [] 01:51:05,166 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from Candidate to Leader 01:51:05,166 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 01:51:05,166 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:05,166 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:05,166 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from Candidate to Leader 01:51:05,167 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionChainWithMultipleShards: Closing 01:51:05,167 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testTransactionChainWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 01:51:05,167 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 01:51:05,167 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:05,167 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:05,167 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-people-testTransactionChainWithMultipleShards status sync done false 01:51:05,167 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:51:05,167 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:05,261 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 01:51:05,261 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 01:51:05,261 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 4 01:51:05,262 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionChainWithMultipleShards 01:51:05,263 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:05,263 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Cluster state: [] 01:51:05,263 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:05,263 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:05,263 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=null 01:51:05,264 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:05,264 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 01:51:05,264 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:05,265 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:05,266 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:05,266 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionChainWithMultipleShards] 01:51:05,266 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:05,266 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:05,266 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:51:05,266 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: became leader with frontend state for [] 01:51:05,266 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from Candidate to Leader 01:51:05,266 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 01:51:05,266 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:05,266 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:05,266 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from Candidate to Leader 01:51:05,266 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All Shards are ready - data store testTransactionChainWithMultipleShards is ready 01:51:05,267 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithMultipleShards: Closing 01:51:05,267 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testTransactionChainWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 01:51:05,267 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 01:51:05,267 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:05,267 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:05,267 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All Shards are ready - data store testTransactionChainWithMultipleShards is ready 01:51:05,268 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-cars-testTransactionChainWithMultipleShards status sync done false 01:51:05,268 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:51:05,268 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:05,283 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-people-testTransactionChainWithMultipleShards status sync done true 01:51:05,317 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#614344334] 01:51:05,317 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: findLocalShard : people 01:51:05,317 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#2003613132] 01:51:05,318 PM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@fc46e84 01:51:05,318 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@fc46e84 changed state from IDLE to TX_OPEN 01:51:05,320 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} 01:51:05,320 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:05,320 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:05,320 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#285640401] 01:51:05,320 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:05,320 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} shard 1 01:51:05,320 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards, primaryVersion=13] 01:51:05,320 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:51:05,320 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards, primaryVersion=13] 01:51:05,321 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:51:05,321 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 01:51:05,321 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} 01:51:05,321 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:51:05,321 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:51:05,321 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#285640401] 01:51:05,321 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:51:05,321 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} shard 2 01:51:05,321 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards, primaryVersion=13] 01:51:05,321 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} scheduling timeout in 30 seconds 01:51:05,321 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:51:05,321 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} allocated sequence 0 01:51:05,321 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards, primaryVersion=13] 01:51:05,322 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@599a8e5d, attempting to connect 01:51:05,322 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@273c7027, attempting to connect 01:51:05,322 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@4037059c 01:51:05,322 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@fbfaee4 01:51:05,322 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@fc46e84 changed state from TX_OPEN to IDLE 01:51:05,322 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@fc46e84 readied transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:51:05,322 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 01:51:05,323 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@6ecb89a5 01:51:05,323 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=FLUSHED} allocated sequence 1 01:51:05,323 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@b115282 01:51:05,332 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} is not yet known 01:51:05,332 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} is not yet known 01:51:05,334 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, nanosAgo=42081, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:51:05,334 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, nanosAgo=41852, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:51:05,338 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:05,338 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:05,338 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-393591239], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:05,338 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-393591239], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:05,338 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@674f597d instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2409a823 01:51:05,338 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:51:05,338 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-393591239], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:05,338 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55832655 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819651434003} 01:51:05,338 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2409a823 01:51:05,338 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:05,338 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} 01:51:05,338 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=2, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} 01:51:05,339 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 01:51:05,341 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2238/0x00000008016aba80@8db6033 01:51:05,341 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55832655 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=2, protocol=READY}, enqueuedTicks=819653293809} 01:51:05,341 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=2, protocol=READY}} 01:51:05,341 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 408 bytes, total slices to send: 1 01:51:05,341 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 01:51:05,341 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} 01:51:05,341 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 1 01:51:05,341 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2239/0x00000008016abcb8@21a19a4b 01:51:05,341 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55832655 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=819653585846} 01:51:05,341 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE}} 01:51:05,341 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 288 bytes, total slices to send: 1 01:51:05,341 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 01:51:05,341 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@fc46e84 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-393591239], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:05,341 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 01:51:05,341 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@509247ef 01:51:05,341 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5b84db4e superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@815a194, splicing queue 01:51:05,341 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5b84db4e queue spliced 0 messages 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-393591239], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 3.505 ms 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#2003613132], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#2003613132], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6b7e1ca9 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@76abda9f 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} previous state FLUSHED 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#2003613132], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@b0821af transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819652266233} 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@76abda9f 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=2, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} allocated sequence 0 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2238/0x00000008016aba80@3810f9bc 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@b0821af transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=2, protocol=READY}, enqueuedTicks=819653360870} 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, message: Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=2, protocol=READY}} 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 416 bytes, total slices to send: 1 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 1 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2239/0x00000008016abcb8@732022aa 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@b0821af transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=819653644407} 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, message: Envelope{sessionId=1, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE}} 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 289 bytes, total slices to send: 1 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@fc46e84 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#2003613132], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@7276b94e 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@145cfbff superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3a7efd6a, splicing queue 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@145cfbff queue spliced 0 messages 01:51:05,342 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#2003613132], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 795.8 μs 01:51:05,345 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 01:51:05,345 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 01:51:05,346 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:51:05,346 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 01:51:05,346 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 01:51:05,346 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:51:05,346 PM [cluster-test-shard-dispatcher-32] [DEBUG] LeaderFrontendState - member-1-shard-people-testTransactionChainWithMultipleShards: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:51:05,346 PM [cluster-test-shard-dispatcher-21] [DEBUG] LeaderFrontendState - member-1-shard-cars-testTransactionChainWithMultipleShards: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:51:05,346 PM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:51:05,346 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:51:05,346 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@7468c8a 01:51:05,346 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:05,346 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:51:05,346 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@55e30c14 01:51:05,346 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:51:05,346 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 ready 01:51:05,346 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE, modificationType=WRITE}}}]} 01:51:05,346 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:05,346 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 to ready 01:51:05,347 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 ready 01:51:05,347 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 is already in state READY (READY) 01:51:05,347 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 initiating canCommit 01:51:05,347 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 01:51:05,347 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:51:05,347 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 to ready 01:51:05,347 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 is already in state READY (READY) 01:51:05,347 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 initiating canCommit 01:51:05,347 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:05,347 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 validated 01:51:05,347 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:05,347 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 completed canCommit 01:51:05,347 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 validated 01:51:05,347 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:05,347 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 completed canCommit 01:51:05,348 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 01:51:05,348 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 01:51:05,351 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=2, protocol=READY} with Envelope{sessionId=1, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0}} 01:51:05,351 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=2, protocol=READY}, enqueuedTicks=819653360870, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0}} 01:51:05,351 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=2, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0} 01:51:05,351 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=2, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0}} 01:51:05,351 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=2, protocol=READY}, enqueuedTicks=819653293809, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0}} 01:51:05,351 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=2, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0} 01:51:05,351 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1}} 01:51:05,351 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=819653585846, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1}} 01:51:05,351 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} canCommit completed 01:51:05,351 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1}} 01:51:05,351 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=819653644407, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1}} 01:51:05,351 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} canCommit completed 01:51:05,351 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:51:05,351 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@1c8e59d 01:51:05,351 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} 01:51:05,352 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 2 01:51:05,352 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x0000000801303580@32f8131f 01:51:05,352 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55832655 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819682751346} 01:51:05,352 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 2 01:51:05,352 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@54006ebc 01:51:05,352 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} 01:51:05,352 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 2 01:51:05,352 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x0000000801303580@12a29dac 01:51:05,352 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@b0821af transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819682902200} 01:51:05,353 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 initiating preCommit 01:51:05,353 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 initiating preCommit 01:51:05,354 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:05,354 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:51:05,354 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 candidate ready 01:51:05,354 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 candidate ready 01:51:05,354 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1: canCommit - candidate rootPath: / 01:51:05,354 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2: canCommit - candidate rootPath: / 01:51:05,354 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1: canCommit - messages: [] 01:51:05,354 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2: canCommit - messages: [] 01:51:05,354 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1: preCommit - successfulFromPrevious: [] 01:51:05,354 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2: preCommit - successfulFromPrevious: [] 01:51:05,354 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 prepared 01:51:05,354 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 prepared 01:51:05,354 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:05,354 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:05,354 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 completed preCommit 01:51:05,354 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 completed preCommit 01:51:05,356 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615]} with Envelope{sessionId=0, txSequence=3, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2}} 01:51:05,356 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819682751346, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2}} 01:51:05,356 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:51:05,356 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615]} with Envelope{sessionId=1, txSequence=3, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2}} 01:51:05,356 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819682902200, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2}} 01:51:05,357 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:51:05,357 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 3 01:51:05,357 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#481258615]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@358c378d 01:51:05,357 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} 01:51:05,357 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 3 01:51:05,357 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#481258615]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@358c378d 01:51:05,357 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55832655 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819687695528} 01:51:05,357 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 3 01:51:05,357 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#481258615]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@622eca0e 01:51:05,357 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} 01:51:05,357 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 3 01:51:05,357 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#481258615]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@622eca0e 01:51:05,357 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@b0821af transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819687819931} 01:51:05,359 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 initiating commit 01:51:05,359 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 initiating commit 01:51:05,359 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:51:05,359 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:05,359 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 181 01:51:05,359 PM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 177 01:51:05,359 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 01:51:05,359 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 01:51:05,359 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 01:51:05,359 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 01:51:05,360 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:05,360 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:05,360 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:51:05,360 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:51:05,360 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 01:51:05,360 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 01:51:05,360 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}]]] 01:51:05,360 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}]]] 01:51:05,360 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 submitted to persistence 01:51:05,360 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 submitted to persistence 01:51:05,360 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 01:51:05,360 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 01:51:05,361 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}]]] 01:51:05,361 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 01:51:05,361 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:05,361 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 01:51:05,361 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 01:51:05,361 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}]]] 01:51:05,361 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 01:51:05,361 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:05,361 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=177} 01:51:05,361 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 01:51:05,361 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 01:51:05,361 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 01:51:05,361 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 2 01:51:05,361 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=181} 01:51:05,361 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 01:51:05,361 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:05,361 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 2 01:51:05,361 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:05,361 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 01:51:05,361 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 01:51:05,362 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-cars-testTransactionChainWithMultipleShards status sync done true 01:51:05,362 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 01:51:05,362 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 01:51:05,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:51:05,362 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:51:05,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 01:51:05,362 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 01:51:05,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 0 01:51:05,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 01:51:05,362 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 0 01:51:05,362 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 01:51:05,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 01:51:05,362 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 01:51:05,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2241/0x00000008016af5d0@2f714f13 01:51:05,362 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2241/0x00000008016af5d0@349beae 01:51:05,362 PM [cluster-test-shard-dispatcher-21] [DEBUG] LeaderFrontendState - member-1-shard-cars-testTransactionChainWithMultipleShards: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:51:05,362 PM [cluster-test-shard-dispatcher-32] [DEBUG] LeaderFrontendState - member-1-shard-people-testTransactionChainWithMultipleShards: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:51:05,363 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionChainWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1a67729d}, purged=MutableUnsignedLongSet{size=0}} 01:51:05,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionChainWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@2b6aca2d}, purged=MutableUnsignedLongSet{size=0}} 01:51:05,363 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:51:05,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:51:05,363 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 1 01:51:05,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 1 01:51:05,363 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 01:51:05,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 01:51:05,363 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 1 data Simple{size=181} 01:51:05,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 1 data Simple{size=177} 01:51:05,363 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:51:05,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:05,363 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:51:05,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:05,363 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2: commit - successfulFromPrevious: [] 01:51:05,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1: commit - successfulFromPrevious: [] 01:51:05,363 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 01:51:05,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:51:05,363 PM [cluster-test-shard-dispatcher-32] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE, modificationType=WRITE}}}]} 01:51:05,363 PM [cluster-test-shard-dispatcher-21] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 01:51:05,364 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-189398547] with name member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher 01:51:05,364 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-35105363] with name member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher 01:51:05,364 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:51:05,364 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:05,364 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:51:05,364 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:51:05,364 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:05,364 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:51:05,364 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-189398547] 01:51:05,364 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-35105363] 01:51:05,364 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:51:05,364 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:51:05,364 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 76.15 μs 01:51:05,364 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-189398547] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:05,364 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-35105363] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:05,364 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 109.8 μs 01:51:05,369 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 1 01:51:05,369 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 01:51:05,369 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 0 01:51:05,369 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 01:51:05,369 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 01:51:05,370 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionChainWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5f44d41a}, purged=MutableUnsignedLongSet{size=0}} 01:51:05,370 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:51:05,370 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 1 01:51:05,370 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 01:51:05,370 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 1 data Simple{size=181} 01:51:05,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 1 01:51:05,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 01:51:05,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 0 01:51:05,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 01:51:05,370 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:51:05,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 01:51:05,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionChainWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5066e840}, purged=MutableUnsignedLongSet{size=0}} 01:51:05,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:51:05,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 1 01:51:05,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 01:51:05,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 1 data Simple{size=177} 01:51:05,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:05,370 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:51:05,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:05,370 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-1153226976] with name member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher 01:51:05,370 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:51:05,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1934118197] with name member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher 01:51:05,370 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:51:05,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:51:05,370 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:51:05,370 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-1153226976] 01:51:05,370 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:51:05,371 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 85.39 μs 01:51:05,371 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-1153226976] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:05,371 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1934118197] 01:51:05,371 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:51:05,371 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} 01:51:05,371 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1934118197] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:05,371 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819651434003, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} 01:51:05,371 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0} 01:51:05,371 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615]} with Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} 01:51:05,371 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 76.32 μs 01:51:05,371 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819652266233, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} 01:51:05,371 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0} 01:51:05,371 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#481258615]} with Envelope{sessionId=1, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3}} 01:51:05,371 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819687819931, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3}} 01:51:05,371 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} doCommit completed 01:51:05,371 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6b7e1ca9 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 01:51:05,371 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR}: initiating purge 01:51:05,371 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 4 01:51:05,371 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#481258615]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@646b8f86 01:51:05,371 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} 01:51:05,372 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED}: initiating purge 01:51:05,372 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} allocated sequence 4 01:51:05,372 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#481258615]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@255685bd 01:51:05,372 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#2003613132], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:05,372 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@b0821af transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819702257457} 01:51:05,372 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#481258615]} with Envelope{sessionId=0, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3}} 01:51:05,372 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819687695528, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3}} 01:51:05,372 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} doCommit completed 01:51:05,372 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@674f597d completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 01:51:05,372 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: initiating purge 01:51:05,372 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 4 01:51:05,372 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@fc46e84 changed state from IDLE to TX_OPEN 01:51:05,372 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#481258615]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@14b9016b 01:51:05,372 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} 01:51:05,372 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED}: initiating purge 01:51:05,372 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 4 01:51:05,372 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#481258615]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@541366a0 01:51:05,372 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-393591239], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:05,372 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55832655 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819702905703} 01:51:05,372 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:51:05,372 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 0 01:51:05,372 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:51:05,372 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} allocated sequence 0 01:51:05,372 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 1 01:51:05,372 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@660a45d1 01:51:05,372 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55832655 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=1}, enqueuedTicks=819703439157} 01:51:05,372 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=6, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=1}} 01:51:05,373 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 452 bytes, total slices to send: 1 01:51:05,373 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 01:51:05,373 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@7a4f8d63 01:51:05,373 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55832655 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]}, enqueuedTicks=819703818436} 01:51:05,381 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:51:05,381 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 01:51:05,381 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 01:51:05,381 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:05,381 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:05,381 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 01:51:05,381 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 01:51:05,381 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:05,381 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:51:05,381 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:51:05,381 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6161aeb4 01:51:05,382 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-569: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1}} 01:51:05,384 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-569: Message size: 371 bytes, total slices to send: 1 01:51:05,384 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-569: Message does not need to be sliced - sending original message 01:51:05,384 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 01:51:05,384 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 01:51:05,386 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=1} with Envelope{sessionId=0, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0}} 01:51:05,386 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=1}, enqueuedTicks=819703439157, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0}} 01:51:05,386 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=1} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0} 01:51:05,392 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]} with Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1}} 01:51:05,392 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]}, enqueuedTicks=819703818436, sessionId=0, txSequence=7} with Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1}} 01:51:05,392 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1} 01:51:05,392 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} allocated sequence 1 01:51:05,392 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@1d41d126 01:51:05,392 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@b0821af transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=1}, enqueuedTicks=819723369248} 01:51:05,392 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, message: Envelope{sessionId=1, txSequence=6, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=1}} 01:51:05,393 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 440 bytes, total slices to send: 1 01:51:05,393 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 01:51:05,393 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@250bc1a6 01:51:05,393 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@b0821af transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]}, enqueuedTicks=819723710327} 01:51:05,414 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:51:05,414 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:51:05,414 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@64bf7c8f 01:51:05,416 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=1} with Envelope{sessionId=1, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0}} 01:51:05,416 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=1}, enqueuedTicks=819723369248, sessionId=1, txSequence=6} with Envelope{sessionId=1, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0}} 01:51:05,416 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=1} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0} 01:51:05,422 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-570: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, message: Envelope{sessionId=1, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1}} 01:51:05,423 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-570: Message size: 359 bytes, total slices to send: 1 01:51:05,423 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-570: Message does not need to be sliced - sending original message 01:51:05,424 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]} with Envelope{sessionId=1, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1}} 01:51:05,424 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]}, enqueuedTicks=819723710327, sessionId=1, txSequence=7} with Envelope{sessionId=1, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1}} 01:51:05,424 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1} 01:51:05,426 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 2 01:51:05,426 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@179eea2e 01:51:05,426 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55832655 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=READY}, enqueuedTicks=819757146992} 01:51:05,426 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=READY}} 01:51:05,426 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 289 bytes, total slices to send: 1 01:51:05,426 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 01:51:05,426 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} allocated sequence 2 01:51:05,426 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@269deb29 01:51:05,426 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@b0821af transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=READY}, enqueuedTicks=819757340697} 01:51:05,426 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, message: Envelope{sessionId=1, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=READY}} 01:51:05,426 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 290 bytes, total slices to send: 1 01:51:05,426 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 01:51:05,426 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@fc46e84 changed state from TX_OPEN to IDLE 01:51:05,426 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@fc46e84 readied transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:51:05,426 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@fc46e84 changed state from IDLE to TX_OPEN 01:51:05,427 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:51:05,427 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=OPEN} allocated sequence 0 01:51:05,427 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@2a15dc2 01:51:05,427 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@b0821af transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=1, protocol=READY}, enqueuedTicks=819757699146} 01:51:05,427 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, message: Envelope{sessionId=1, txSequence=9, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=1, protocol=READY}} 01:51:05,427 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 426 bytes, total slices to send: 1 01:51:05,427 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 01:51:05,427 PM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:51:05,427 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@fc46e84 changed state from TX_OPEN to IDLE 01:51:05,427 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@fc46e84 readied transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:51:05,427 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} allocated sequence 3 01:51:05,427 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@255aaa17 01:51:05,430 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2409a823 skipping transactions [2] 01:51:05,432 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55832655 transmitting entry ConnectionEntry{request=SkipTransactionsRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819762619197} 01:51:05,432 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55832655 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=819762695189} 01:51:05,432 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=a, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE}} 01:51:05,432 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 289 bytes, total slices to send: 1 01:51:05,432 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 01:51:05,432 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} allocated sequence 3 01:51:05,432 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@34bf3385 01:51:05,432 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@b0821af transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=819762862423} 01:51:05,432 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, message: Envelope{sessionId=1, txSequence=a, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE}} 01:51:05,432 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 290 bytes, total slices to send: 1 01:51:05,432 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 01:51:05,439 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:51:05,440 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 ready 01:51:05,440 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:51:05,440 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 to ready 01:51:05,441 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} skipping transactions [[2..2]] 01:51:05,442 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 01:51:05,442 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 01:51:05,442 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.SkipTransactionsPayload, isSendImmediate: false 01:51:05,442 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 is already in state READY (READY) 01:51:05,442 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 initiating canCommit 01:51:05,442 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:51:05,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 validated 01:51:05,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:05,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 completed canCommit 01:51:05,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 01:51:05,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:51:05,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 ready 01:51:05,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}]} 01:51:05,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 to ready 01:51:05,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:51:05,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeTransactionChain - Reusing a chained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:51:05,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 snapshot MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}] 01:51:05,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:51:05,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 ready 01:51:05,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=DELETE}}}}}}}]} 01:51:05,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 to ready 01:51:05,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 is already in state READY (READY) 01:51:05,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 initiating canCommit 01:51:05,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:51:05,444 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 validated 01:51:05,444 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:05,444 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 completed canCommit 01:51:05,444 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=READY} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2}} 01:51:05,444 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=READY}, enqueuedTicks=819757146992, sessionId=0, txSequence=8} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2}} 01:51:05,444 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2} 01:51:05,445 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=a, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3}} 01:51:05,445 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=819762695189, sessionId=0, txSequence=10} with Envelope{sessionId=0, txSequence=a, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3}} 01:51:05,445 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} canCommit completed 01:51:05,445 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=READY} with Envelope{sessionId=1, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2}} 01:51:05,445 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=READY}, enqueuedTicks=819757340697, sessionId=1, txSequence=8} with Envelope{sessionId=1, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2}} 01:51:05,445 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2} 01:51:05,445 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=1, protocol=READY} with Envelope{sessionId=1, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0}} 01:51:05,445 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=1, protocol=READY}, enqueuedTicks=819757699146, sessionId=1, txSequence=9} with Envelope{sessionId=1, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0}} 01:51:05,445 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0} 01:51:05,445 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=a, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3}} 01:51:05,445 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=819762862423, sessionId=1, txSequence=10} with Envelope{sessionId=1, txSequence=a, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3}} 01:51:05,445 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} canCommit completed 01:51:05,445 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} allocated sequence 4 01:51:05,445 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#481258615]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@6f616078 01:51:05,446 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55832655 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819776540311} 01:51:05,446 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} allocated sequence 4 01:51:05,446 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#481258615]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@25c3629a 01:51:05,446 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@b0821af transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819776612923} 01:51:05,446 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 initiating preCommit 01:51:05,446 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:51:05,446 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 candidate ready 01:51:05,446 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2: canCommit - candidate rootPath: / 01:51:05,447 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2: canCommit - messages: [] 01:51:05,447 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2: preCommit - successfulFromPrevious: [] 01:51:05,447 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 prepared 01:51:05,447 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:05,447 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 completed preCommit 01:51:05,447 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 initiating preCommit 01:51:05,447 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:51:05,447 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 candidate ready 01:51:05,447 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1: canCommit - candidate rootPath: / 01:51:05,447 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1: canCommit - messages: [] 01:51:05,447 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1: preCommit - successfulFromPrevious: [] 01:51:05,447 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 prepared 01:51:05,447 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:05,447 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 completed preCommit 01:51:05,447 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#481258615]} with Envelope{sessionId=1, txSequence=b, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4}} 01:51:05,447 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819776612923, sessionId=1, txSequence=11} with Envelope{sessionId=1, txSequence=b, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4}} 01:51:05,447 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} preCommit completed, clearing successfulRequests 01:51:05,447 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#481258615]} with Envelope{sessionId=0, txSequence=b, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4}} 01:51:05,447 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819776540311, sessionId=0, txSequence=11} with Envelope{sessionId=0, txSequence=b, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4}} 01:51:05,447 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} preCommit completed, clearing successfulRequests 01:51:05,447 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} allocated sequence 5 01:51:05,448 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#481258615]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@119f97e5 01:51:05,448 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55832655 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819778571041} 01:51:05,448 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} allocated sequence 5 01:51:05,448 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#481258615]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@7519470e 01:51:05,448 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@b0821af transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819778633713} 01:51:05,448 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 initiating commit 01:51:05,448 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:51:05,448 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 236 01:51:05,448 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 01:51:05,448 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 01:51:05,448 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:05,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: 0 01:51:05,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 01:51:05,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}], SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}], SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}]]] 01:51:05,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 submitted to persistence 01:51:05,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 initiating commit 01:51:05,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:51:05,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 225 01:51:05,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 01:51:05,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 01:51:05,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:05,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 01:51:05,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 01:51:05,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}]]] 01:51:05,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 submitted to persistence 01:51:05,449 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 01:51:05,449 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 01:51:05,450 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}], SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}], SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}]]] 01:51:05,450 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 3 01:51:05,450 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:51:05,450 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69} 01:51:05,450 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 01:51:05,450 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75} 01:51:05,450 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 01:51:05,450 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=236} 01:51:05,450 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 01:51:05,450 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 4 01:51:05,451 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:05,451 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}]]] 01:51:05,451 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 01:51:05,451 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:51:05,451 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69} 01:51:05,451 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 01:51:05,451 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=225} 01:51:05,451 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 01:51:05,451 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 3 01:51:05,451 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:05,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 3, nextIndex: 4 01:51:05,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 01:51:05,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 2 01:51:05,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 01:51:05,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69} 01:51:05,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@73707de8 01:51:05,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:51:05,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:51:05,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 3 01:51:05,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 01:51:05,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=225} 01:51:05,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:51:05,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:51:05,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2: commit - successfulFromPrevious: [] 01:51:05,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 changing state from COMMIT_PENDING to COMMITTED 01:51:05,452 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 4, nextIndex: 5 01:51:05,452 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 1 01:51:05,452 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 2 01:51:05,452 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 01:51:05,452 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69} 01:51:05,452 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@5bf5700c 01:51:05,452 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:05,452 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:05,452 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 3 01:51:05,452 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 01:51:05,452 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 3 data SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75} 01:51:05,452 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2254/0x00000008016bc220@325fb81d 01:51:05,452 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: History LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} skipped transactions ImmutableUnsignedLongSet{span=[2..2], size=1} 01:51:05,452 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 4 01:51:05,452 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 01:51:05,452 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 4 data Simple{size=236} 01:51:05,452 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:51:05,452 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:51:05,452 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1: commit - successfulFromPrevious: [] 01:51:05,452 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 01:51:05,452 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 01:51:05,452 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 01:51:05,452 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 01:51:05,452 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 01:51:05,452 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 01:51:05,453 PM [cluster-test-shard-dispatcher-21] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}]} 01:51:05,453 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 2 for term 4 01:51:05,453 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:51:05,453 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=3 01:51:05,453 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:51:05,453 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 48.32 μs 01:51:05,453 PM [cluster-test-shard-dispatcher-31] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:51:05,453 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 4 01:51:05,453 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:51:05,453 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 01:51:05,453 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:51:05,453 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 28.40 μs 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#481258615]} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4}} 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819702257457, sessionId=1, txSequence=5} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4}} 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE}: purge completed 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@76abda9f purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR}: purge completed 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6b7e1ca9 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#481258615]} with Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4}} 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819702905703, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4}} 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2409a823 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: purge completed 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@674f597d purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request SkipTransactionsRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615]} with Envelope{sessionId=0, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0}} 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=SkipTransactionsRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819762619197, sessionId=0, txSequence=9} with Envelope{sessionId=0, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0}} 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2409a823 confirmed transaction skip 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#481258615]} with Envelope{sessionId=1, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5}} 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819778633713, sessionId=1, txSequence=12} with Envelope{sessionId=1, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5}} 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} doCommit completed 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@76abda9f completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED}: initiating purge 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=DONE} allocated sequence 6 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#481258615]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@45797af2 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#2003613132], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@b0821af transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819787207974} 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#481258615]} with Envelope{sessionId=0, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5}} 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819778571041, sessionId=0, txSequence=12} with Envelope{sessionId=0, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5}} 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} doCommit completed 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2409a823 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED}: initiating purge 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=DONE} allocated sequence 6 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#481258615]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@fe5e11 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-393591239], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:05,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55832655 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819787378528} 01:51:05,456 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 3 01:51:05,456 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 01:51:05,456 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 2 01:51:05,456 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 01:51:05,456 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69} 01:51:05,457 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 01:51:05,457 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 3 01:51:05,457 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 01:51:05,457 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=225} 01:51:05,457 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:51:05,457 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 4 01:51:05,457 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 1, lastIndex(): 4 01:51:05,457 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 2 01:51:05,457 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 01:51:05,457 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69} 01:51:05,457 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:05,457 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 3 01:51:05,457 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 01:51:05,457 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 3 data SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75} 01:51:05,457 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:51:05,457 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: History LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} skipped transactions ImmutableUnsignedLongSet{span=[2..2], size=1} 01:51:05,457 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 4 01:51:05,457 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 01:51:05,457 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 4 data Simple{size=236} 01:51:05,457 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 2 for term 4 01:51:05,457 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=3 01:51:05,457 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:51:05,457 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 28.21 μs 01:51:05,457 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:51:05,457 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=FLUSHED} allocated sequence 1 01:51:05,457 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@68d8a6c5 01:51:05,457 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@b0821af transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=SIMPLE}, enqueuedTicks=819788420374} 01:51:05,457 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, message: Envelope{sessionId=1, txSequence=e, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=SIMPLE}} 01:51:05,457 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message size: 290 bytes, total slices to send: 1 01:51:05,457 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-571: Message does not need to be sliced - sending original message 01:51:05,458 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:51:05,458 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:51:05,458 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 31.99 μs 01:51:05,458 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 4 01:51:05,458 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 01:51:05,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:51:05,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 01:51:05,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 01:51:05,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:05,460 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:51:05,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 is already in state READY (READY) 01:51:05,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 initiating direct canCommit 01:51:05,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:51:05,460 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 01:51:05,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 validated 01:51:05,460 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 01:51:05,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:05,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 initiating direct preCommit 01:51:05,460 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:05,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:51:05,462 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 candidate ready 01:51:05,462 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2: canCommit - candidate rootPath: / 01:51:05,462 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2: canCommit - messages: [] 01:51:05,462 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2: preCommit - successfulFromPrevious: [] 01:51:05,462 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 prepared 01:51:05,462 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:05,462 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 initiating direct commit 01:51:05,462 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:51:05,463 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 01:51:05,463 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 01:51:05,463 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 01:51:05,463 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:05,463 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 5, leaderSnapShotIndex: 2 01:51:05,463 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 4 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 01:51:05,463 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}], SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}]]] 01:51:05,463 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 submitted to persistence 01:51:05,463 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}], SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}]]] 01:51:05,463 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 01:51:05,463 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 01:51:05,463 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70} 01:51:05,463 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 01:51:05,463 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=222} 01:51:05,463 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 01:51:05,463 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 3 01:51:05,463 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=5, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:05,464 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 01:51:05,464 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 01:51:05,464 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 5, nextIndex: 6 01:51:05,464 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 2 01:51:05,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 01:51:05,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 01:51:05,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 01:51:05,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 4 01:51:05,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 01:51:05,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70} 01:51:05,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@1801fc7b 01:51:05,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:51:05,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:51:05,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 4 01:51:05,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 01:51:05,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 01:51:05,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 01:51:05,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 5 01:51:05,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 4 to 5 01:51:05,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 5 data Simple{size=222} 01:51:05,469 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:51:05,469 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:51:05,469 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2: commit - successfulFromPrevious: [] 01:51:05,469 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 changing state from COMMIT_PENDING to COMMITTED 01:51:05,469 PM [cluster-test-shard-dispatcher-21] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, modificationType=DISAPPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=DELETE, modificationType=DELETE}}}}}}}]} 01:51:05,469 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 4 for term 4 01:51:05,469 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=5 01:51:05,469 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:51:05,469 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 52.15 μs 01:51:05,470 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#481258615]} with Envelope{sessionId=1, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6}} 01:51:05,470 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819787207974, sessionId=1, txSequence=13} with Envelope{sessionId=1, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6}} 01:51:05,470 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=DONE}: purge completed 01:51:05,470 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@76abda9f purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=DONE} 01:51:05,470 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=SIMPLE} with Envelope{sessionId=1, txSequence=e, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1}} 01:51:05,470 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=SIMPLE}, enqueuedTicks=819788420374, sessionId=1, txSequence=14} with Envelope{sessionId=1, txSequence=e, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1}} 01:51:05,470 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=FLUSHED} directCommit completed 01:51:05,470 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=FLUSHED}: initiating purge 01:51:05,470 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=DONE} allocated sequence 2 01:51:05,470 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@4f170ec8 01:51:05,470 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#2003613132], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:05,470 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@b0821af transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819801443706} 01:51:05,471 PM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 01:51:05,471 PM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 01:51:05,471 PM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 01:51:05,471 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2409a823 invoking destroy 01:51:05,471 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55832655 transmitting entry ConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819801691142} 01:51:05,471 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@76abda9f invoking destroy 01:51:05,471 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@b0821af transmitting entry ConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819801755073} 01:51:05,471 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@fc46e84 changed state from IDLE to CLOSED 01:51:05,471 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@610b3a66: creating a new snapshot member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 01:51:05,471 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0} shard 1 01:51:05,471 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 01:51:05,471 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:05,471 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@46b49001 01:51:05,471 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55832655 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=819802123272} 01:51:05,474 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:51:05,474 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 01:51:05,474 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 01:51:05,474 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:05,474 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: closing history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:51:05,474 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: closing history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:51:05,475 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:51:05,475 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 01:51:05,475 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 01:51:05,475 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CloseLocalHistoryPayload, isSendImmediate: false 01:51:05,475 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: allocating new open snapshot member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1 01:51:05,476 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-569: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=f, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0}} 01:51:05,476 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 01:51:05,476 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-569: Message size: 370 bytes, total slices to send: 1 01:51:05,476 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-569: Message does not need to be sliced - sending original message 01:51:05,476 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:51:05,476 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 01:51:05,476 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 01:51:05,476 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.CloseLocalHistoryPayload, isSendImmediate: false 01:51:05,476 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 01:51:05,477 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 01:51:05,478 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=f, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0}} 01:51:05,478 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=819802123272, sessionId=0, txSequence=15} with Envelope{sessionId=0, txSequence=f, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0}} 01:51:05,478 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0} 01:51:05,478 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0} shard 2 01:51:05,478 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 01:51:05,478 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:51:05,478 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@5a07b9ed 01:51:05,478 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@b0821af transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]}, enqueuedTicks=819809108845} 01:51:05,479 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: allocating new open snapshot member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2 01:51:05,479 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-570: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=11, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0}} 01:51:05,479 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-570: Message size: 231 bytes, total slices to send: 1 01:51:05,479 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-570: Message does not need to be sliced - sending original message 01:51:05,479 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]} with Envelope{sessionId=1, txSequence=11, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0}} 01:51:05,479 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#481258615], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]}, enqueuedTicks=819809108845, sessionId=1, txSequence=17} with Envelope{sessionId=1, txSequence=11, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0}} 01:51:05,479 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0} 01:51:05,479 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:51:05,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testTransactionChainWithMultipleShards 01:51:05,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testTransactionChainWithMultipleShards 01:51:05,480 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithMultipleShards - waiting on 2 shards 01:51:05,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onShutDown 01:51:05,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Initiating leader transfer 01:51:05,480 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onShutDown 01:51:05,480 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Initiating leader transfer 01:51:05,480 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@cf53d1f 01:51:05,480 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@433bd191 01:51:05,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:51:05,482 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:51:05,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@cf53d1f 01:51:05,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Attempting to transfer leadership 01:51:05,482 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@433bd191 01:51:05,482 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Attempting to transfer leadership 01:51:05,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 6, leaderSnapShotIndex: 3 01:51:05,482 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 6, leaderLastIndex: 7, leaderSnapShotIndex: 4 01:51:05,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 01:51:05,482 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 6 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 01:51:05,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}], SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}]]] 01:51:05,482 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}], SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}]]] 01:51:05,482 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:51:05,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:05,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:05,482 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:51:05,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:05,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:05,483 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}], SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}]]] 01:51:05,483 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}], SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}]]] 01:51:05,483 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 01:51:05,483 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 01:51:05,483 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 01:51:05,483 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 5, entries to be added from: 0 01:51:05,483 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70} 01:51:05,483 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70} 01:51:05,483 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 01:51:05,483 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 01:51:05,484 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 01:51:05,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 01:51:05,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 01:51:05,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 5 01:51:05,484 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 01:51:05,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 5 01:51:05,484 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 3 01:51:05,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=7, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:05,484 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:05,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 3, lastIndex(): 7 01:51:05,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 4 01:51:05,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 01:51:05,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70} 01:51:05,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 01:51:05,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 5 01:51:05,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 4 to 5 01:51:05,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 5 data Simple{size=222} 01:51:05,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:51:05,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:51:05,484 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 01:51:05,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 4 for term 4 01:51:05,484 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 67.76 μs 01:51:05,485 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 01:51:05,485 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 01:51:05,485 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 01:51:05,485 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 01:51:05,485 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards] 01:51:05,485 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=5 01:51:05,485 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards] 01:51:05,485 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:51:05,485 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:05,485 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:05,485 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:51:05,485 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:05,485 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55832655 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@188b0893, splicing queue 01:51:05,486 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-393591239], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:05,486 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55832655 queue spliced 2 messages 01:51:05,486 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-393591239], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-393591239], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:05,489 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: refreshing backend for shard 1 01:51:05,489 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-393591239], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:05,489 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:05,489 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:05,489 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#285640401] 01:51:05,489 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:05,489 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:51:05,489 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 01:51:05,489 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@b0821af superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@71d00164, splicing queue 01:51:05,489 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#2003613132], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:51:05,489 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@b0821af queue spliced 2 messages 01:51:05,489 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#2003613132], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#2003613132], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:05,489 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: refreshing backend for shard 2 01:51:05,489 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#2003613132], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:51:05,489 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:51:05,489 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:51:05,490 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#285640401] 01:51:05,490 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:51:05,490 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:51:05,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 7, nextIndex: 8 01:51:05,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 5 01:51:05,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 6 01:51:05,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 5 to 6 01:51:05,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 6 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70} 01:51:05,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@58ec118a 01:51:05,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:51:05,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:51:05,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 7 01:51:05,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 6 to 7 01:51:05,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 7 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 01:51:05,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2255/0x00000008016bd008@295896e7 01:51:05,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:51:05,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 6 for term 4 01:51:05,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 01:51:05,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testTransactionChainWithMultipleShards, matchIndex: 7, lastIndex: 7, isVoting: true 01:51:05,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:51:05,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 01:51:05,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Leader transfer complete 01:51:05,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=7 01:51:05,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 6, nextIndex: 7 01:51:05,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 4 01:51:05,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 5 01:51:05,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 4 to 5 01:51:05,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70} 01:51:05,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@32cf1126 01:51:05,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:51:05,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:51:05,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 6 01:51:05,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 5 to 6 01:51:05,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 6 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 01:51:05,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2255/0x00000008016bd008@727d30e6 01:51:05,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:51:05,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 5 for term 4 01:51:05,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 01:51:05,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionChainWithMultipleShards, matchIndex: 6, lastIndex: 6, isVoting: true 01:51:05,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:51:05,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 01:51:05,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Leader transfer complete 01:51:05,498 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=6 01:51:05,498 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=f, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2}} not found in inflight queue, checking pending queue 01:51:05,498 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615]} with Envelope{sessionId=1, txSequence=f, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2}} 01:51:05,498 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819801443706, sessionId=1, txSequence=15} with Envelope{sessionId=1, txSequence=f, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2}} 01:51:05,499 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=DONE}: purge completed 01:51:05,499 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@76abda9f purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=DONE} 01:51:05,499 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=10, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1}} not found in inflight queue, checking pending queue 01:51:05,499 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615]} with Envelope{sessionId=1, txSequence=10, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1}} 01:51:05,499 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819801755073, sessionId=1, txSequence=16} with Envelope{sessionId=1, txSequence=10, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1}} 01:51:05,499 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@76abda9f destroy completed with LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1} 01:51:05,499 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@fc46e84: removed destroyed proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@76abda9f 01:51:05,500 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#2003613132], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:05,500 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#2003613132], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:05,501 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 7 01:51:05,501 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 6 01:51:05,501 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 4, lastIndex(): 6 01:51:05,501 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 7, context.getLastApplied(): 5, lastIndex(): 7 01:51:05,501 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 5 01:51:05,501 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 4 to 5 01:51:05,501 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 6 01:51:05,501 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 5 to 6 01:51:05,501 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70} 01:51:05,501 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 6 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70} 01:51:05,501 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 01:51:05,501 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 6 01:51:05,501 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 5 to 6 01:51:05,501 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 6 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 01:51:05,501 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 01:51:05,501 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 01:51:05,501 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 7 01:51:05,501 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 5 for term 4 01:51:05,501 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 6 to 7 01:51:05,501 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 7 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 01:51:05,501 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 01:51:05,501 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 6 for term 4 01:51:05,501 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=6 01:51:05,501 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=7 01:51:05,502 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6}} not found in inflight queue, checking pending queue 01:51:05,502 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#481258615]} with Envelope{sessionId=0, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6}} 01:51:05,502 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819787378528, sessionId=0, txSequence=13} with Envelope{sessionId=0, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6}} 01:51:05,502 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=DONE}: purge completed 01:51:05,502 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2409a823 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=DONE} 01:51:05,502 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=e, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1}} not found in inflight queue, checking pending queue 01:51:05,502 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615]} with Envelope{sessionId=0, txSequence=e, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1}} 01:51:05,502 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615]}, enqueuedTicks=819801691142, sessionId=0, txSequence=14} with Envelope{sessionId=0, txSequence=e, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1}} 01:51:05,502 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2409a823 destroy completed with LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1} 01:51:05,502 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@fc46e84: removed destroyed proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2409a823 01:51:05,502 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-393591239], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:05,502 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-393591239], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:05,892 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All shards shutdown - sending PoisonPill to self 01:51:05,892 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] ShardManager - shard-manager-testTransactionChainWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-393591239]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:51:05,892 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionChainWithMultipleShards 01:51:05,892 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader)_slicer-id-575: Closing 01:51:05,893 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 01:51:05,893 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-1-shard-people-testTransactionChainWithMultipleShards 01:51:05,893 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards (Leader)_slicer-id-574: Closing 01:51:05,893 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 01:51:05,893 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithMultipleShards 01:51:05,893 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 01:51:05,894 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 01:51:05,894 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 01:51:05,894 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: shutting down 01:51:05,894 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 01:51:05,895 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#614344334] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#614344334] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:05,896 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:05,896 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:05,900 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:05,901 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:05,901 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:05,901 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:05,914 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithMultipleShards: Closing 01:51:05,915 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionChainWithMultipleShards 01:51:05,915 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Closing 01:51:05,915 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-people-testTransactionChainWithMultipleShards 01:51:05,915 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 01:51:05,915 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Closing 01:51:05,915 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 01:51:05,915 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithMultipleShards 01:51:05,916 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:05,916 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:06,910 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-3512659926226480510] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:06,930 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [7796815374138719843] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:06,935 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:06,935 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:06,936 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:06.938] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:06,947 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:06,947 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:06,954 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:06.955] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:06,967 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:06,976 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-929152645710311373] 01:51:06,977 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:51:06,978 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:51:06,978 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:51:06,983 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:06,984 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:06,993 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:51:06,994 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:51:06,994 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:51:07,3 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:07,8 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [2832846430846313148] 01:51:07,9 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:51:07,10 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:07,10 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:51:07,11 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:07,11 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:07,26 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:07,30 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2949255738095360616] 01:51:07,30 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:51:07,38 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:07,38 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:51:07,39 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:07,39 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:07,40 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:07,42 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:07,44 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testInstallSnapshot 01:51:07,68 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#1511111498] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:07,69 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: no initial generation override, starting from 0 01:51:07,69 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 01:51:07,69 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5cf47e19 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2bbe71b1 01:51:07,69 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4c7c318f 01:51:07,70 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testInstallSnapshot, 0, 1710165067070, None)) 01:51:07,70 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165067069,0,0)) 01:51:07,71 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2bbe71b1 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@31e6c862 01:51:07,71 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} started 01:51:07,71 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:07,73 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:07,74 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testInstallSnapshot 01:51:07,82 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testInstallSnapshot 01:51:07,84 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testInstallSnapshot 01:51:07,84 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Creating local shard: member-1-shard-cars-testInstallSnapshot 01:51:07,84 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@c908daa 01:51:07,84 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:07,84 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testInstallSnapshot 01:51:07,85 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-1893030454] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:07,89 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: no initial generation override, starting from 0 01:51:07,89 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} 01:51:07,89 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3d2bb70c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4521178a 01:51:07,89 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@18bff753 01:51:07,89 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testInstallSnapshot, 0, 1710165067089, None)) 01:51:07,89 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:07,89 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:51:07,89 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:51:07,89 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165067088,0,0)) 01:51:07,89 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4521178a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@645928dd 01:51:07,89 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testInstallSnapshot, persistent : true 01:51:07,89 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#447114173] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:07,90 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} started 01:51:07,90 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testInstallSnapshot with journal batch size 1 01:51:07,90 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testInstallSnapshot 01:51:07,90 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/shard-member-1:snapshot-read#-168675314] 01:51:07,90 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-notifier#-17387694 created and ready for shard:member-1-shard-cars-testInstallSnapshot 01:51:07,90 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-notifier#-17387694] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:07,90 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/shard-member-1:snapshot-read#-168675314] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:07,91 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.SnapshotOffer from Actor[akka://cluster-test/system/in-memory-snapshot-store#-640202648] 01:51:07,91 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: SnapshotOffer called. 01:51:07,91 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Moving last applied index from -1 to 5 01:51:07,91 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=1, votedFor=null 01:51:07,91 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Applying recovered snapshot 01:51:07,91 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-testInstallSnapshot: clearing clients {} 01:51:07,91 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testInstallSnapshot 01:51:07,91 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Creating local shard: member-2-shard-cars-testInstallSnapshot 01:51:07,92 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testInstallSnapshot: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#961582776] with name member-1-shard-cars-testInstallSnapshot-DTCL-publisher 01:51:07,92 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#961582776] 01:51:07,92 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testInstallSnapshot: publishChanges: rootPath: / 01:51:07,92 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testInstallSnapshot: state snapshot applied in 935.4 μs 01:51:07,92 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Recovery snapshot applied for member-1-shard-cars-testInstallSnapshot in 1.087 ms: snapshotIndex=5, snapshotTerm=1, journal-size=0 01:51:07,92 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testInstallSnapshot: Elapsed time for generation of change events for member-1-shard-cars-testInstallSnapshot-DTCL-publisher: 183.3 μs 01:51:07,92 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#961582776] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:07,92 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:07,92 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Recovery completed in 1.780 ms - Switching actor to Follower - last log index = 5, last log term = 1, snapshot index = 5, snapshot term = 1, journal size = 0 01:51:07,93 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 01:51:07,93 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testInstallSnapshot is no longer the leader 01:51:07,93 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from null to Follower 01:51:07,93 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Initializing shard [cars] 01:51:07,93 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testInstallSnapshot is initialized 01:51:07,93 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , registered listener akka://cluster-test/user/shardmanager-testInstallSnapshot 01:51:07,93 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2fd1da19 01:51:07,93 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:07,93 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testInstallSnapshot 01:51:07,93 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testInstallSnapshot, persistent : true 01:51:07,94 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot#-225574166] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:07,94 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testInstallSnapshot with journal batch size 1 01:51:07,95 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from null to Follower 01:51:07,95 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-notifier#-60021656 created and ready for shard:member-2-shard-cars-testInstallSnapshot 01:51:07,95 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-notifier#-60021656] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:07,96 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:07,96 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:07,96 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 01:51:07,96 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testInstallSnapshot is no longer the leader 01:51:07,96 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testInstallSnapshot , received role change from null to Follower 01:51:07,96 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Initializing shard [cars] 01:51:07,96 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testInstallSnapshot is initialized 01:51:07,96 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testInstallSnapshot , registered listener akka://cluster-test/user/shardmanager-testInstallSnapshot 01:51:07,96 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-2-shard-cars-testInstallSnapshot from null to Follower 01:51:07,97 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/shard-member-2:snapshot-read#1477535762] 01:51:07,97 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/shard-member-2:snapshot-read#1477535762] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:07,103 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:51:07,108 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:07,110 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:51:07,112 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:07,112 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:51:07,112 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:51:07,112 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-1-shard-cars-testInstallSnapshot set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:51:07,112 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:07,140 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#1511111498] 01:51:07,140 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: findLocalShard : cars 01:51:07,141 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#447114173] 01:51:07,345 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:07,345 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Follower): Received ElectionTimeout - switching to Candidate 01:51:07,345 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testInstallSnapshot] 01:51:07,345 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=2, votedFor=member-1-shard-cars-testInstallSnapshot 01:51:07,345 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 2 01:51:07,345 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Follower) :- Switching from behavior Follower to Candidate, election term: 2 01:51:07,345 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testInstallSnapshot (Follower): Closing 01:51:07,345 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 01:51:07,345 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testInstallSnapshot is no longer the leader 01:51:07,345 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from Follower to Candidate 01:51:07,345 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from Follower to Candidate 01:51:07,595 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 01:51:07,595 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=3, votedFor=member-1-shard-cars-testInstallSnapshot 01:51:07,595 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 3 01:51:07,635 PM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:07,874 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 01:51:07,874 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=4, votedFor=member-1-shard-cars-testInstallSnapshot 01:51:07,874 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 4 01:51:07,995 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:51:07,995 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:51:07,996 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:07,996 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot 01:51:07,996 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:51:07,996 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:07,996 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:51:07,996 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:51:07,996 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-2-shard-cars-testInstallSnapshot set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:51:08,39 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:08,39 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot 01:51:08,39 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:51:08,39 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:08,39 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:51:08,39 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 01:51:08,54 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:51:08,175 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 01:51:08,175 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 01:51:08,175 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 5 01:51:08,175 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] to peer member-2-shard-cars-testInstallSnapshot 01:51:08,177 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:08,177 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Cluster state: [] 01:51:08,177 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] with higher term appears reachable, updating term. 01:51:08,177 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Term 5 in "RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1]" message is greater than follower's term 0 - updating term 01:51:08,177 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=null 01:51:08,177 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): In requestVote: RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:08,177 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 01:51:08,177 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:51:08,179 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:51:08,179 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:51:08,179 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Election: Leader has following peers: [member-2-shard-cars-testInstallSnapshot] 01:51:08,179 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: false, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 5 01:51:08,179 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:51:08,179 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: true, hasLeader : true 01:51:08,179 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: became leader with frontend state for [] 01:51:08,179 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from Candidate to Leader 01:51:08,179 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testInstallSnapshot, leaderId=member-1-shard-cars-testInstallSnapshot, leaderPayloadVersion=13] 01:51:08,180 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:08,180 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:08,180 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from Candidate to Leader 01:51:08,180 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testInstallSnapshot: All Shards are ready - data store testInstallSnapshot is ready 01:51:08,184 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testInstallSnapshot: Closing 01:51:08,184 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: getLeaderAddress leaderId = member-1-shard-cars-testInstallSnapshot peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 01:51:08,184 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testInstallSnapshot, leaderId=member-1-shard-cars-testInstallSnapshot, leaderPayloadVersion=13] 01:51:08,184 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:08,184 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:08,184 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: All Shards are ready - data store testInstallSnapshot is ready 01:51:08,184 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot Received follower initial sync status for member-2-shard-cars-testInstallSnapshot status sync done false 01:51:08,185 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testInstallSnapshot updated: matchIndex: -1, nextIndex: 0 01:51:08,185 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 0, leaderLastIndex: 5, leaderSnapShotIndex: 5 01:51:08,185 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Initiating install snapshot to follower member-2-shard-cars-testInstallSnapshot: follower nextIndex: 0, leader snapshotIndex: 5, leader lastIndex: 5, leader log size: 0 01:51:08,185 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex 5 and lastAppliedTerm 1 instead. 01:51:08,185 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=5, lastAppliedTerm=1, lastIndex=5, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-testInstallSnapshot 01:51:08,185 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: lastSequenceNumber prior to capture: 4 01:51:08,185 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 01:51:08,186 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: CaptureSnapshotReply received by actor 01:51:08,186 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Persisting of snapshot done: Snapshot [lastIndex=5, lastTerm=1, lastAppliedIndex=5, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7aaec345, electionTerm=5, electionVotedFor=member-1-shard-cars-testInstallSnapshot, ServerConfigPayload=null] 01:51:08,186 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 5 and term: 1 01:51:08,186 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): sendInstallSnapshot 01:51:08,186 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): next snapshot chunk size for follower member-2-shard-cars-testInstallSnapshot: 376 01:51:08,186 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-testInstallSnapshot, member-2-shard-cars-testInstallSnapshot), Chunk: 1/1 01:51:08,186 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: SaveSnapshotSuccess received for snapshot, sequenceNr: 4 01:51:08,186 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Snapshot success - sequence number: 4 01:51:08,186 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: SnapshotComplete received 01:51:08,187 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: handled a snapshot response 01:51:08,187 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testInstallSnapshot Received follower initial sync status for member-2-shard-cars-testInstallSnapshot status sync done true 01:51:08,187 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: handled a journal response 01:51:08,187 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): handleInstallSnapshot: InstallSnapshot [term=5, leaderId=member-1-shard-cars-testInstallSnapshot, lastIncludedIndex=5, lastIncludedTerm=1, datasize=376, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 01:51:08,187 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 01:51:08,187 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Snapshot installed from leader: member-1-shard-cars-testInstallSnapshot 01:51:08,187 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Applying snapshot on follower: Snapshot [lastIndex=5, lastTerm=1, lastAppliedIndex=5, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5497dd12, electionTerm=5, electionVotedFor=member-1-shard-cars-testInstallSnapshot, ServerConfigPayload=null] 01:51:08,187 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - lastSequenceNumber prior to persisting applied snapshot: 2 01:51:08,188 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: SaveSnapshotSuccess received for snapshot, sequenceNr: 2 01:51:08,188 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Snapshot success - sequence number: 2 01:51:08,188 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Moving last applied index from -1 to 5 01:51:08,188 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 01:51:08,188 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Applying snapshot 01:51:08,188 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-testInstallSnapshot: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 01:51:08,188 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testInstallSnapshot: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#1821133584] with name member-2-shard-cars-testInstallSnapshot-DTCL-publisher 01:51:08,189 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testInstallSnapshot: state snapshot applied in 258.4 μs 01:51:08,189 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Done applying snapshot 01:51:08,189 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=5, followerId=member-2-shard-cars-testInstallSnapshot, chunkIndex=1, success=true] 01:51:08,189 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: SnapshotComplete received 01:51:08,189 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#1821133584] 01:51:08,189 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testInstallSnapshot: publishChanges: rootPath: / 01:51:08,189 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#1821133584] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:08,189 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testInstallSnapshot: Elapsed time for generation of change events for member-2-shard-cars-testInstallSnapshot-DTCL-publisher: 115.3 μs 01:51:08,189 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: handled a snapshot response 01:51:08,189 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: handled a journal response 01:51:08,190 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=5, followerId=member-2-shard-cars-testInstallSnapshot, chunkIndex=1, success=true] 01:51:08,190 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Snapshot successfully installed on follower member-2-shard-cars-testInstallSnapshot (last chunk 1) - matchIndex set to 5, nextIndex set to 6 01:51:08,197 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@14d99f12: creating a new snapshot member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 01:51:08,197 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} 01:51:08,197 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:08,197 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:08,197 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#1511111498] 01:51:08,197 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, history=0, cookie=0} shard 1 01:51:08,197 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testInstallSnapshot-fe-0-txn-0-1 for transaction member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 01:51:08,197 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:08,197 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1470019548], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@544a07e 01:51:08,197 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testInstallSnapshot: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:51:08,197 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:08,197 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#447114173, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@51d73792] 01:51:08,197 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#447114173, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@51d73792] 01:51:08,197 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@4290160c, attempting to connect 01:51:08,198 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: client ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} is not yet known 01:51:08,198 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, nanosAgo=7070, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 01:51:08,198 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:51:08,198 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#447114173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:51:08,198 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#447114173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:08,198 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@72207169 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@7c93e5c6 01:51:08,198 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:51:08,198 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@7c93e5c6 01:51:08,198 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testInstallSnapshot-fe-0-txn-0-1 for transaction member-1-datastore-testInstallSnapshot-fe-0-txn-0-1 01:51:08,198 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} 01:51:08,198 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1470019548], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} 01:51:08,198 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@14d99f12 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#447114173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:08,198 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} 01:51:08,198 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@25c25a9d 01:51:08,198 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1b83969f superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@78ddfdfc, splicing queue 01:51:08,198 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1b83969f queue spliced 0 messages 01:51:08,198 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#447114173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 342.3 μs 01:51:08,198 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=0} 01:51:08,198 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 01:51:08,199 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Issuing gracefulStop to shard member-1-shard-cars-testInstallSnapshot 01:51:08,199 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-testInstallSnapshot - waiting on 1 shards 01:51:08,199 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onShutDown 01:51:08,199 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Initiating leader transfer 01:51:08,199 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2b1078dd 01:51:08,199 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testInstallSnapshot: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2b1078dd 01:51:08,199 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Attempting to transfer leadership 01:51:08,199 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 5 01:51:08,199 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testInstallSnapshot, leaderId=null, leaderPayloadVersion=13] 01:51:08,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:08,199 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:08,199 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@47fdeffd superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@692cbfb5, splicing queue 01:51:08,199 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@47fdeffd queue spliced 0 messages 01:51:08,199 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#447114173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#447114173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:08,199 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: refreshing backend for shard 1 01:51:08,199 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#447114173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:51:08,199 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:08,199 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:08,199 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#1511111498] 01:51:08,200 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:08,200 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testInstallSnapshot, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 01:51:08,200 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testInstallSnapshot] 01:51:08,200 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testInstallSnapshot, leaderId=null, leaderPayloadVersion=13] 01:51:08,200 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:08,200 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:08,201 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testInstallSnapshot, matchIndex: 5, lastIndex: 5, isVoting: true 01:51:08,201 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Follower's log matches - sending ElectionTimeout 01:51:08,201 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 5 01:51:08,201 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Leader transfer complete 01:51:08,417 PM [cluster-test-shard-dispatcher-11] [WARN] TestShard - member-1-shard-cars-testInstallSnapshot (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-2-shard-cars-testInstallSnapshot, logLastIndex=5, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 214, lastApplied : 5, commitIndex : 5 01:51:08,615 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Received ShardNotInitializedTimeout message for shard cars 01:51:08,615 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Returning NoShardLeaderException for shard cars 01:51:08,615 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: All shards shutdown - sending PoisonPill to self 01:51:08,615 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] ShardManager - shard-manager-testInstallSnapshot: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#447114173]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:51:08,615 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testInstallSnapshot currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:51:08,616 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testInstallSnapshot currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:08,616 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testInstallSnapshot currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:08,616 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testInstallSnapshot currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:08,617 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-1-shard-cars-testInstallSnapshot 01:51:08,617 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testInstallSnapshot (Leader)_slicer-id-580: Closing 01:51:08,617 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Aborting 0 pending queued transactions 01:51:08,617 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-testInstallSnapshot 01:51:08,617 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 01:51:08,617 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 01:51:08,618 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 01:51:08,618 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#1511111498] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#1511111498] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:08,618 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 01:51:08,619 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:08,619 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:08,623 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:08,623 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:08,623 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:08,623 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:08,626 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:51:08,635 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testInstallSnapshot: Closing 01:51:08,636 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testInstallSnapshot 01:51:08,636 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testInstallSnapshot (Follower): Closing 01:51:08,636 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Aborting 0 pending queued transactions 01:51:08,636 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testInstallSnapshot 01:51:08,637 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:08,637 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:09,634 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-929152645710311373] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:09,654 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [2832846430846313148] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:09,657 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:09,658 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:09,659 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:09.660] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:09,666 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:09,666 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:09,668 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:09.669] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:09,679 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:09,685 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [1688397369950521201] 01:51:09,685 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:51:09,686 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:51:09,686 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:51:09,687 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:09,687 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:09,698 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:51:09,698 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:51:09,698 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:51:09,699 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:09,703 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [8962979319735190963] 01:51:09,703 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:51:09,704 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:09,704 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:51:09,705 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:09,705 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:09,728 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:09,739 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [9151175198483091853] 01:51:09,739 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:51:09,742 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:09,742 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:51:09,742 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:09,742 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:09,754 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 closeTransaction called 01:51:09,754 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_35#-199199217)] 01:51:09,754 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 closeTransaction called 01:51:09,754 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_34#1721284947)] 01:51:09,754 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@ef8416c aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} 01:51:09,754 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 1 01:51:09,754 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@2ba5997b 01:51:09,754 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#-393591239], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:09,754 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@9cabe51 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} 01:51:09,754 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 1 01:51:09,754 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#481258615], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@2562627f 01:51:09,754 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#2003613132], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:09,754 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0} 01:51:09,754 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@72207169 aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} 01:51:09,754 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:51:09,754 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1470019548], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@215df20b 01:51:09,754 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testInstallSnapshot: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} 01:51:09,755 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@7c93e5c6 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} 01:51:09,755 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1470019548]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@645f6844 01:51:09,755 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testInstallSnapshot: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#447114173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 01:51:09,755 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testInstallSnapshot: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#447114173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:51:09,755 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:51:09,761 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:09,763 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:09,764 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:09,766 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 01:51:09,769 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1929435166] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:09,772 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:51:09,773 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 01:51:09,773 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:51:09,773 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1496949e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@444576b3 01:51:09,773 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@407afae3 01:51:09,773 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 01:51:09,773 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1710165069773, None)) 01:51:09,774 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:09,774 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:09,774 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:09,774 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165069772,0,0)) 01:51:09,774 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5af272a7 01:51:09,774 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@444576b3 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1396a923 01:51:09,774 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 01:51:09,775 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:09,776 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:09,778 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 01:51:09,779 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:09,779 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:09,780 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:09,780 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:09,780 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#467397051] 01:51:09,781 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Shard created : member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 01:51:09,781 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#467397051] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:09,781 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-42009190] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:09,781 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-501173332 created and ready for shard:member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:09,782 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-501173332] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:09,782 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-1:snapshot-read#1383520347] 01:51:09,782 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-1:snapshot-read#1383520347] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:09,782 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:51:09,785 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 01:51:09,786 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:09,786 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:09,786 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:51:09,786 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 01:51:09,786 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from null to Follower 01:51:09,786 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Initializing shard [cars] 01:51:09,786 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is initialized 01:51:09,786 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , registered listener akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader 01:51:09,801 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:51:09,802 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:09,805 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from null to Follower 01:51:09,809 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 01:51:09,809 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 01:51:09,809 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:51:09,809 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:09,809 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1900e103 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@642ac2b1 01:51:09,809 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@54b67d5b 01:51:09,809 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1710165069809, None)) 01:51:09,809 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:09,809 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:09,809 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165069808,0,0)) 01:51:09,809 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@642ac2b1 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4f6c6985 01:51:09,809 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 01:51:09,810 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7ebbfb36 01:51:09,810 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:09,810 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:09,810 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:09,810 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:09,810 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#-471526932] 01:51:09,810 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Shard created : member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 01:51:09,811 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#-471526932] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:09,811 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-1955986956 created and ready for shard:member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:09,811 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 01:51:09,811 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-1955986956] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:09,812 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-2:snapshot-read#-84046655] 01:51:09,812 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-2:snapshot-read#-84046655] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:09,812 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:09,812 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:09,812 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:51:09,812 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 01:51:09,812 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Initializing shard [cars] 01:51:09,812 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader is initialized 01:51:09,812 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from null to Follower 01:51:09,812 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:09,813 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader , registered listener akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader 01:51:09,813 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader from null to Follower 01:51:09,860 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1929435166] 01:51:09,860 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 01:51:09,860 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#467397051] 01:51:10,28 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:10,28 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Received ElectionTimeout - switching to Candidate 01:51:10,28 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] 01:51:10,28 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,28 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 1 01:51:10,28 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:10,28 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Closing 01:51:10,28 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:51:10,28 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 01:51:10,28 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from Follower to Candidate 01:51:10,28 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from Follower to Candidate 01:51:10,257 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 01:51:10,257 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,257 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 2 01:51:10,388 PM [cluster-test-shard-dispatcher-19] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:10,527 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 01:51:10,527 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,527 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 3 01:51:10,699 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:10,699 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,699 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,699 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:10,699 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,699 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,699 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,699 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,699 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:51:10,699 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:51:10,704 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:10,704 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,704 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,704 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:10,705 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,705 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,705 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,756 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:51:10,837 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 01:51:10,837 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,837 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 4 01:51:10,837 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,838 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,839 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:10,839 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [] 01:51:10,839 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:10,839 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:10,839 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=null 01:51:10,839 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:10,839 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,839 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#467397051] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:10,839 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:10,840 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:10,841 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:10,841 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Election: Leader has following peers: [member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] 01:51:10,841 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:10,841 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:10,841 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:10,841 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:51:10,841 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: became leader with frontend state for [] 01:51:10,841 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from Candidate to Leader 01:51:10,841 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderPayloadVersion=13] 01:51:10,841 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:10,841 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:10,841 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from Candidate to Leader 01:51:10,841 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 01:51:10,842 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Closing 01:51:10,842 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: getLeaderAddress leaderId = member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,842 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderPayloadVersion=13] 01:51:10,842 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:10,842 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:10,842 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 01:51:10,842 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader Received follower initial sync status for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader status sync done false 01:51:10,842 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader updated: matchIndex: -1, nextIndex: 0 01:51:10,843 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:10,843 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#467397051] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:10,864 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:10,866 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:10,869 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 01:51:10,872 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#1668742344] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:10,873 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:51:10,875 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 01:51:10,875 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:51:10,875 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 01:51:10,875 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,875 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2d358e23 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@376363aa 01:51:10,875 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6e904513 01:51:10,875 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:10,875 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1710165070875, None)) 01:51:10,875 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,875 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:10,875 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,875 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:10,875 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,875 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165070874,0,0)) 01:51:10,876 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@376363aa to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@28708054 01:51:10,876 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@75fa80ff 01:51:10,876 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 01:51:10,876 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:10,876 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,876 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,876 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,876 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#1075369500] 01:51:10,876 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,876 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,876 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#1075369500] 01:51:10,876 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,876 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,876 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#1075369500] 01:51:10,876 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:51:10,877 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Shard created : member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 01:51:10,877 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-3:snapshot-read#-1365535448] 01:51:10,877 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#853550124 created and ready for shard:member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,877 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Starting recovery for member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 01:51:10,877 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Stopping Shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:10,877 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 01:51:10,877 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:51:10,878 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:10,878 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:10,882 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:10,882 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:10,882 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:10,882 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:10,884 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:51:10,897 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1929435166] 01:51:10,897 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 01:51:10,897 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#467397051] 01:51:10,897 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#467397051] 01:51:10,959 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader Received follower initial sync status for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader status sync done true 01:51:11,719 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:51:11,898 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [9151175198483091853] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:11,899 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [9151175198483091853] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:16,819 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 01:51:16,819 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:51:16,819 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:16,840 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 01:51:16,840 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:16,921 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-42009190] 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 01:51:16,922 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#-471526932] 01:51:16,922 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#467397051] 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): In requestVote: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 01:51:16,922 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 01:51:16,923 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#467397051] to Actor[akka://cluster-test/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:16,923 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#467397051] to Actor[akka://cluster-test/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:16,923 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#467397051] 01:51:16,923 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#-471526932] to Actor[akka://cluster-test/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:16,923 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#-471526932] to Actor[akka://cluster-test/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:16,923 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:51:16,923 PM [main] [WARN] ActorUtils - ShardManager for testSemiReachableCandidateNotDroppingLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.testSemiReachableCandidateNotDroppingLeader(DistributedDataStoreRemotingIntegrationTest.java:1322) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:51:16,924 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:51:16,924 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:51:16,924 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Issuing gracefulStop to shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:16,924 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader - waiting on 1 shards 01:51:16,924 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onShutDown 01:51:16,924 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Initiating leader transfer 01:51:16,925 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@268052fa 01:51:16,925 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@268052fa 01:51:16,925 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Attempting to transfer leadership 01:51:16,925 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:16,925 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:16,925 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 01:51:16,925 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:16,925 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:16,931 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader] 01:51:16,931 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 01:51:16,931 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:16,931 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:16,932 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, matchIndex: -1, lastIndex: -1, isVoting: true 01:51:16,932 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Follower's log matches - sending ElectionTimeout 01:51:16,932 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:16,932 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:16,932 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Leader transfer complete 01:51:17,338 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All shards shutdown - sending PoisonPill to self 01:51:17,338 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#467397051]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:51:17,338 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:17,338 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader)_slicer-id-585: Closing 01:51:17,339 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 01:51:17,339 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:51:17,339 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:51:17,339 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 01:51:17,339 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: shutting down 01:51:17,339 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:51:17,339 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:51:17,340 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#718902844] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#718902844] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:17,340 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1929435166] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1929435166] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:17,340 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#718902844] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#718902844] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:17,340 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1929435166] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1929435166] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:17,340 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:17,340 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:17,918 PM [cluster-test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Closing 01:51:17,918 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 01:51:18,358 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [1688397369950521201] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:18,359 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:18,359 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:18,360 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:18.362] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:18,370 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 01:51:18,370 PM [cluster-test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Closing 01:51:18,370 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 01:51:18,370 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 01:51:18,371 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 01:51:18,371 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:18,371 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/11/2024 13:51:18.374] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:18,383 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:18,389 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4419475859367592272] 01:51:18,389 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:51:18,390 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:51:18,390 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:51:18,391 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:18,391 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:18,401 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:51:18,401 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:51:18,401 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:51:18,404 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:51:18,413 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-4782205518671646609] 01:51:18,413 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:51:18,419 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:18,419 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:51:18,425 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:18,425 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:18,433 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:18,443 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2398673420789182399] 01:51:18,443 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:51:18,444 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:18,444 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:51:18,455 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:18,456 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:18,457 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadyLocalTransactionForwardedToLeader 01:51:18,460 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:18,460 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:18,464 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 01:51:18,464 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-994567902] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:18,469 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testReadyLocalTransactionForwardedToLeader 01:51:18,469 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: no initial generation override, starting from 0 01:51:18,469 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 01:51:18,469 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Creating local shard: member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:18,469 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@27c5a866 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@80cfba1 01:51:18,470 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7f350303 01:51:18,470 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:18,470 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:18,470 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, 0, 1710165078470, None)) 01:51:18,470 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165078469,0,0)) 01:51:18,470 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@80cfba1 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2f1a4ea2 01:51:18,470 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} started 01:51:18,471 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:18,472 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:18,473 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadyLocalTransactionForwardedToLeader 01:51:18,474 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@22a5e1f3 01:51:18,474 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:18,474 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:18,483 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 01:51:18,483 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:18,483 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:18,483 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: updated peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068] 01:51:18,483 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, persistent : true 01:51:18,484 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with journal batch size 1 01:51:18,484 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:18,484 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-1:snapshot-read#-439875562] 01:51:18,487 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#646554819] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:18,487 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#735569373] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:18,488 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#646554819 created and ready for shard:member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:18,490 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:18,490 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:18,491 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadyLocalTransactionForwardedToLeader 01:51:18,491 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Creating local shard: member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:18,491 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: no initial generation override, starting from 0 01:51:18,491 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 01:51:18,491 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@587ff0a0 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@720a597f 01:51:18,491 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3708d95a 01:51:18,491 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, 0, 1710165078491, None)) 01:51:18,492 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-1:snapshot-read#-439875562] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:18,493 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:51:18,493 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 01:51:18,493 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Initializing shard [cars] 01:51:18,493 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is initialized 01:51:18,493 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from null to Follower 01:51:18,493 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader 01:51:18,493 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from null to Follower 01:51:18,494 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165078490,0,0)) 01:51:18,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1d5a6670 01:51:18,494 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@720a597f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1b6980a2 01:51:18,495 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} started 01:51:18,495 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:18,495 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:18,498 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, persistent : true 01:51:18,498 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#1861843191] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:18,498 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-2:snapshot-read#1480104519] 01:51:18,498 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-2:snapshot-read#1480104519] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:18,498 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with journal batch size 1 01:51:18,498 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#-876942008 created and ready for shard:member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:18,498 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#-876942008] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:18,499 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:18,499 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:18,499 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:51:18,499 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Initializing shard [cars] 01:51:18,499 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 01:51:18,499 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader is initialized 01:51:18,500 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from null to Follower 01:51:18,500 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader 01:51:18,500 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader from null to Follower 01:51:18,512 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:51:18,520 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:18,523 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:18,524 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:18,524 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:18,524 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:18,524 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:18,524 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:51:18,545 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-994567902] 01:51:18,546 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 01:51:18,546 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068] 01:51:18,802 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:18,802 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Received ElectionTimeout - switching to Candidate 01:51:18,802 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] 01:51:18,802 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:18,802 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 1 01:51:18,802 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:18,802 PM [cluster-test-shard-dispatcher-16] [DEBUG] MessageAssembler - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Closing 01:51:18,802 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:51:18,802 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 01:51:18,803 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from Follower to Candidate 01:51:18,803 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from Follower to Candidate 01:51:19,83 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:19,91 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:51:19,91 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,92 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 2 01:51:19,352 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:51:19,352 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,352 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 3 01:51:19,403 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:51:19,403 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:51:19,403 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:19,403 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,403 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,403 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:19,403 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,403 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,403 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,457 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:19,457 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,457 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,457 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:19,457 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,457 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,468 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:51:19,642 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:51:19,642 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,642 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 4 01:51:19,642 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:19,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Cluster state: [] 01:51:19,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:19,643 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:19,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=null 01:51:19,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:19,644 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,644 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:19,645 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:19,645 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:19,645 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] 01:51:19,645 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:19,645 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:19,645 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:51:19,645 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: became leader with frontend state for [] 01:51:19,645 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from Candidate to Leader 01:51:19,645 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderPayloadVersion=13] 01:51:19,645 PM [cluster-test-shard-dispatcher-16] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:19,645 PM [cluster-test-shard-dispatcher-16] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:19,645 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from Candidate to Leader 01:51:19,645 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All Shards are ready - data store testReadyLocalTransactionForwardedToLeader is ready 01:51:19,646 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Closing 01:51:19,646 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,646 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader status sync done false 01:51:19,646 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderPayloadVersion=13] 01:51:19,646 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:19,646 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:19,646 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All Shards are ready - data store testReadyLocalTransactionForwardedToLeader is ready 01:51:19,647 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: -1, nextIndex: 0 01:51:19,647 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:19,705 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#735569373] 01:51:19,705 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 01:51:19,705 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#1861843191] 01:51:19,705 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,705 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#735569373] 01:51:19,705 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 01:51:19,705 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#1861843191] 01:51:19,717 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: handleReadyLocalTransaction for member-1-ShardTransactionTest-fe-0-txn-245-0 01:51:19,717 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,717 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Forwarding ReadyLocalTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader)] 01:51:19,720 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, disabling transaction tracking 01:51:19,720 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:51:19,720 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:51:19,720 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:51:19,720 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:19,720 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,720 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 01:51:19,720 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-245-0 01:51:19,721 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-245-0 of 1 operations, client version 13 01:51:19,721 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-245-0 01:51:19,721 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-245-0 ready 01:51:19,721 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-245-0 01:51:19,721 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 validated 01:51:19,721 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:19,721 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-245-0: success 01:51:19,721 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-245-0 01:51:19,721 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-245-0 01:51:19,721 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 candidate ready 01:51:19,721 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-245-0: canCommit - candidate rootPath: / 01:51:19,721 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-245-0: canCommit - messages: [] 01:51:19,721 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-245-0: preCommit - successfulFromPrevious: [] 01:51:19,721 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 prepared 01:51:19,721 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:19,721 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-245-0 01:51:19,721 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-245-0 01:51:19,721 PM [cluster-test-shard-dispatcher-16] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 197 01:51:19,721 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:51:19,721 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:51:19,722 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-245-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:19,722 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:51:19,722 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,722 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 submitted to persistence 01:51:19,722 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 01:51:19,722 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:51:19,722 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:19,722 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:51:19,722 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:51:19,722 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 1 01:51:19,722 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:19,723 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:51:19,723 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader status sync done true 01:51:19,723 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:51:19,724 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:51:19,724 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 0, nextIndex: 1 01:51:19,724 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:51:19,724 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 0 01:51:19,724 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from -1 to 0 01:51:19,724 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:51:19,724 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 01:51:19,724 PM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 01:51:19,724 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, nanosAgo=3988089} is already disabled 01:51:19,724 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:51:19,724 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,724 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 01:51:19,724 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 01:51:19,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 01:51:19,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:51:19,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:51:19,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log Simple{size=197} 01:51:19,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:51:19,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 2 01:51:19,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 0 01:51:19,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:19,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:51:19,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 0 01:51:19,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from -1 to 0 01:51:19,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:51:19,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 01:51:19,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 01:51:19,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 01:51:19,725 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 01:51:19,726 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 1, nextIndex: 2 01:51:19,726 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:51:19,726 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 1 01:51:19,726 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 0 to 1 01:51:19,726 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 01:51:19,726 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-245-0 01:51:19,726 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-245-0: commit - successfulFromPrevious: [] 01:51:19,726 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 changing state from COMMIT_PENDING to COMMITTED 01:51:19,726 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-245-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/system/testProbe-764#7416919] 01:51:19,726 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#963159378] with name member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher 01:51:19,726 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#963159378] 01:51:19,727 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 01:51:19,727 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 01:51:19,727 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:19,727 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#963159378] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:19,727 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 01:51:19,727 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 197.8 μs 01:51:19,736 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 1 01:51:19,736 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:51:19,736 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5ad28284: creating a new snapshot member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 01:51:19,736 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 1 01:51:19,737 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 0 to 1 01:51:19,737 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 01:51:19,737 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} 01:51:19,737 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:19,737 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:19,737 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-994567902] 01:51:19,737 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, history=0, cookie=0} shard 1 01:51:19,737 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-245-0 01:51:19,737 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1 for transaction member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 01:51:19,737 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:19,737 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#870307478], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@1a7ae761 01:51:19,737 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:51:19,737 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:19,737 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@1cd662d0] 01:51:19,737 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#1360857478] with name member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher 01:51:19,737 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 01:51:19,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 01:51:19,738 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#1360857478] 01:51:19,738 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 01:51:19,738 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#1360857478] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:19,738 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 124.6 μs 01:51:19,738 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@1cd662d0] 01:51:19,739 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@791a99e8, attempting to connect 01:51:19,739 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: client ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} is not yet known 01:51:19,739 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, nanosAgo=7290, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 01:51:19,739 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:51:19,744 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:51:19,744 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:19,744 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4e71f8e4 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5afe29fd 01:51:19,744 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:51:19,744 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5afe29fd 01:51:19,744 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1 for transaction member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1 01:51:19,744 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 01:51:19,744 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#870307478], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 01:51:19,745 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5ad28284 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:19,745 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} 01:51:19,745 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@6b2c9af4 01:51:19,745 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@752ae93b superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@5eb097a2, splicing queue 01:51:19,745 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@752ae93b queue spliced 0 messages 01:51:19,745 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 505.6 μs 01:51:19,745 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0} 01:51:19,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: handleReadyLocalTransaction for member-1-ShardTransactionTest-fe-0-txn-246-0 01:51:19,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Forwarding ReadyLocalTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader)] 01:51:19,749 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-246-0 01:51:19,749 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-246-0 of 1 operations, client version 13 01:51:19,749 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-246-0 01:51:19,749 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-246-0 ready 01:51:19,751 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0 canCommit 01:51:19,751 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068)] resolved 01:51:19,751 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: successfully resolved all cohort actors 01:51:19,751 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0 finishCanCommit 01:51:19,751 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: sending CanCommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-246-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068)] 01:51:19,751 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068)] 01:51:19,751 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Can committing transaction member-1-ShardTransactionTest-fe-0-txn-246-0 01:51:19,751 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-246-0 01:51:19,751 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 validated 01:51:19,751 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:19,751 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-246-0: success 01:51:19,751 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:51:19,751 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: canCommit returning result: true 01:51:19,751 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0 commit 01:51:19,751 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0 finish commit 01:51:19,751 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: Sending CommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-246-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068)] 01:51:19,751 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068)] 01:51:19,752 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-246-0 01:51:19,752 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-246-0 01:51:19,752 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 candidate ready 01:51:19,752 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-246-0: canCommit - candidate rootPath: / 01:51:19,752 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-246-0: canCommit - messages: [] 01:51:19,752 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-246-0: preCommit - successfulFromPrevious: [] 01:51:19,752 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 prepared 01:51:19,752 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:19,752 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-246-0 01:51:19,752 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-246-0 01:51:19,752 PM [cluster-test-shard-dispatcher-16] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 208 01:51:19,752 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:51:19,752 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:51:19,752 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-246-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:19,753 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:51:19,753 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,753 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 01:51:19,753 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 submitted to persistence 01:51:19,753 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:51:19,754 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 01:51:19,754 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:51:19,754 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:51:19,754 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log Simple{size=208} 01:51:19,754 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:51:19,754 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 2 01:51:19,754 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:19,754 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 01:51:19,761 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 2, nextIndex: 3 01:51:19,761 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:51:19,761 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 2 01:51:19,761 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 1 to 2 01:51:19,761 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 01:51:19,761 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-246-0 01:51:19,761 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-246-0: commit - successfulFromPrevious: [] 01:51:19,761 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 changing state from COMMIT_PENDING to COMMITTED 01:51:19,761 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-246-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testReadyLocalTransactionForwardedToLeader_member-1-shard-cars-testReadyLocalTransactionForwardedToLeader%23-809207068$a] 01:51:19,762 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 01:51:19,762 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:51:19,762 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 01:51:19,762 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-246-0: commit succeeded 01:51:19,763 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 01:51:19,763 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 93.04 μs 01:51:19,764 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5ad28284: creating a new snapshot member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0 01:51:19,764 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-1 for transaction member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0 01:51:19,764 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 01:51:19,765 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Issuing gracefulStop to shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:19,765 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader - waiting on 1 shards 01:51:19,765 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onShutDown 01:51:19,765 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Initiating leader transfer 01:51:19,765 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 01:51:19,765 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@54642510 01:51:19,765 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:19,765 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@54642510 01:51:19,765 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Attempting to transfer leadership 01:51:19,765 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 2 01:51:19,765 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:51:19,765 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:51:19,765 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 2 01:51:19,765 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 1 to 2 01:51:19,765 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 01:51:19,765 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-246-0 01:51:19,766 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 01:51:19,766 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 01:51:19,766 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 01:51:19,767 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:19,767 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@aa28c7a superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@73e86883, splicing queue 01:51:19,767 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@aa28c7a queue spliced 0 messages 01:51:19,767 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:19,767 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: refreshing backend for shard 1 01:51:19,767 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:51:19,767 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:19,767 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:19,767 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-994567902] 01:51:19,767 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:19,767 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 01:51:19,768 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, matchIndex: 2, lastIndex: 2, isVoting: true 01:51:19,768 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Follower's log matches - sending ElectionTimeout 01:51:19,768 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 233.8 μs 01:51:19,768 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:51:19,768 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Leader transfer complete 01:51:19,769 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader] 01:51:19,769 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 01:51:19,769 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:19,769 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:20,182 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 01:51:20,182 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received ShardNotInitializedTimeout message for shard cars 01:51:20,182 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Returning NoShardLeaderException for shard cars 01:51:20,182 PM [cluster-test-akka.actor.default-dispatcher-19] [WARN] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:51:20,182 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:51:20,183 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:20,183 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:20,183 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:20,183 PM [cluster-test-shard-dispatcher-16] [DEBUG] MessageSlicer - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader)_slicer-id-592: Closing 01:51:20,183 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Aborting 0 pending queued transactions 01:51:20,183 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 01:51:20,184 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 01:51:20,183 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:20,184 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 01:51:20,184 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 01:51:20,184 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: shutting down 01:51:20,184 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 01:51:20,185 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:20,185 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:20,190 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:20,190 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:51:20.190] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-917)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:51:20.190] [cluster-test-akka.remote.default-remote-dispatcher-8] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-917)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:20,205 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 01:51:20,206 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 01:51:20,206 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Closing 01:51:20,206 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Aborting 0 pending queued transactions 01:51:20,206 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 01:51:20,206 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:20,206 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:21,199 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4419475859367592272] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:21,218 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-4782205518671646609] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:21,224 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:21,224 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:21,228 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:21.229] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:21,244 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:21,244 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/11/2024 13:51:21.246] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:21,269 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:21,279 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [3557717566302733594] 01:51:21,279 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:51:21,279 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:51:21,279 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:51:21,281 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:21,281 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:21,290 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:51:21,290 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:51:21,290 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:51:21,292 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:51:21,297 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [5198425423885922202] 01:51:21,297 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:51:21,298 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:21,298 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:51:21,299 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:21,299 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:21,314 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:21,323 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-9204062398498050499] 01:51:21,323 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:51:21,325 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:21,325 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:51:21,334 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:51:21,334 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:21,334 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:21,335 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:21,336 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:21,339 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:21,340 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithSingleShard 01:51:21,343 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithSingleShard 01:51:21,347 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#846173846] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:21,356 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithSingleShard 01:51:21,356 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Creating local shard: member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:51:21,356 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:21,356 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:51:21,364 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: no initial generation override, starting from 0 01:51:21,364 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:51:21,364 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@71b0fb90 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@60af9222 01:51:21,364 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@25ad0053 01:51:21,365 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, 0, 1710165081364, None)) 01:51:21,365 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165081363,0,0)) 01:51:21,365 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@60af9222 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@8e0d9cb 01:51:21,365 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@521a96d8 01:51:21,365 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} started 01:51:21,366 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:21,368 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:21,370 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithSingleShard 01:51:21,371 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:21,371 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:51:21,378 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:51:21,378 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:51:21,378 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testReadWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard on shard member-1-shard-cars-testReadWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#2103460874] 01:51:21,378 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testReadWriteTransactionWithSingleShard, persistent : true 01:51:21,378 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#2103460874] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:21,379 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-notifier#478698912 created and ready for shard:member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:51:21,379 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadWriteTransactionWithSingleShard with journal batch size 1 01:51:21,379 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-notifier#478698912] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:21,379 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-1:snapshot-read#-1507974571] 01:51:21,379 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-1:snapshot-read#-1507974571] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:21,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:21,380 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:21,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:51:21,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 01:51:21,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Initializing shard [cars] 01:51:21,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is initialized 01:51:21,381 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from null to Follower 01:51:21,383 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard 01:51:21,387 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithSingleShard 01:51:21,389 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from null to Follower 01:51:21,391 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-28977174] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:21,393 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: no initial generation override, starting from 0 01:51:21,393 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:51:21,393 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3861722e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@16197faf 01:51:21,393 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@531be308 01:51:21,393 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, 0, 1710165081393, None)) 01:51:21,393 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165081392,0,0)) 01:51:21,393 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@16197faf to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6d830da7 01:51:21,393 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} started 01:51:21,396 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithSingleShard 01:51:21,397 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:51:21,397 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:21,397 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:51:21,397 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@b073028 01:51:21,397 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:21,397 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:51:21,397 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:51:21,397 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:51:21,397 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testReadWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard on shard member-2-shard-cars-testReadWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard#1040922355] 01:51:21,397 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testReadWriteTransactionWithSingleShard, persistent : true 01:51:21,398 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadWriteTransactionWithSingleShard with journal batch size 1 01:51:21,398 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard#1040922355] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:21,398 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-notifier#761936133 created and ready for shard:member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:51:21,398 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-notifier#761936133] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:21,398 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-477677814] 01:51:21,398 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-477677814] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:21,399 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:51:21,400 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:21,401 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:21,401 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:21,401 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:51:21,401 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 01:51:21,402 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithSingleShard , received role change from null to Follower 01:51:21,402 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Initializing shard [cars] 01:51:21,402 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard is initialized 01:51:21,402 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:51:21,402 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard 01:51:21,402 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testReadWriteTransactionWithSingleShard from null to Follower 01:51:21,444 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#846173846] 01:51:21,444 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: findLocalShard : cars 01:51:21,444 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#2103460874] 01:51:21,663 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:21,663 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 01:51:21,663 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadWriteTransactionWithSingleShard] 01:51:21,663 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:51:21,664 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 1 01:51:21,664 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:21,664 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Closing 01:51:21,664 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:51:21,664 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 01:51:21,664 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from Follower to Candidate 01:51:21,664 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from Follower to Candidate 01:51:21,934 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:51:21,934 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:51:21,934 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 2 01:51:22,10 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:22,234 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:51:22,234 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:51:22,234 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 3 01:51:22,294 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:51:22,294 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:51:22,295 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:22,295 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard 01:51:22,295 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:51:22,295 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:22,295 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:51:22,295 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:51:22,295 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:51:22,306 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:22,306 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard 01:51:22,306 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:51:22,306 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:22,306 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:51:22,306 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:51:22,353 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:51:22,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:51:22,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:51:22,493 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 4 01:51:22,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:51:22,495 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:22,495 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Cluster state: [] 01:51:22,495 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:22,495 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:22,495 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 01:51:22,495 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:22,495 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:51:22,495 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:22,495 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:22,495 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:22,496 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadWriteTransactionWithSingleShard] 01:51:22,496 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:22,496 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:22,496 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:51:22,496 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: became leader with frontend state for [] 01:51:22,496 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Closing 01:51:22,496 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testReadWriteTransactionWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:51:22,496 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithSingleShard status sync done false 01:51:22,496 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:51:22,496 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:22,496 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:22,497 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All Shards are ready - data store testReadWriteTransactionWithSingleShard is ready 01:51:22,497 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: -1, nextIndex: 0 01:51:22,497 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:22,499 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4e2d0d47: creating a new transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:51:22,499 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} 01:51:22,499 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:22,499 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:22,499 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-28977174] 01:51:22,499 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:22,499 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard, primaryVersion=13] 01:51:22,499 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} shard 1 01:51:22,499 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:51:22,499 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:22,499 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 1 01:51:22,499 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=3} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@4fb1226c 01:51:22,499 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:51:22,499 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-499438950], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@6f0fd232 01:51:22,500 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard, primaryVersion=13] 01:51:22,500 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@472c54fe, attempting to connect 01:51:22,500 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} is not yet known 01:51:22,501 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, nanosAgo=5521, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:51:22,501 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:22,501 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#2103460874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:22,501 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#2103460874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:22,501 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@c326824 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@12973f7f 01:51:22,501 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:51:22,501 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@12973f7f 01:51:22,502 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:22,502 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 01:51:22,502 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=3} to successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 01:51:22,502 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:22,502 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-499438950], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 01:51:22,502 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} enqueing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=3} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2271/0x00000008016c4688@46aebd05 01:51:22,502 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#2103460874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:22,502 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6d691e68 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=3}, enqueuedTicks=836830534051} 01:51:22,502 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from Candidate to Leader 01:51:22,502 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:51:22,502 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-595: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=3}} 01:51:22,502 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:22,502 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:22,502 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from Candidate to Leader 01:51:22,502 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All Shards are ready - data store testReadWriteTransactionWithSingleShard is ready 01:51:22,502 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-595: Message size: 467 bytes, total slices to send: 1 01:51:22,502 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-595: Message does not need to be sliced - sending original message 01:51:22,502 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 1 01:51:22,502 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-499438950], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1247/0x0000000801301400@5da3f231 01:51:22,502 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6d691e68 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-499438950], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=836830534051} 01:51:22,502 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4e2d0d47 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#2103460874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:22,502 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 01:51:22,502 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@e45729a 01:51:22,502 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6927a565 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@5b66851b, splicing queue 01:51:22,502 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6927a565 queue spliced 0 messages 01:51:22,503 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#2103460874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.127 ms 01:51:22,504 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithSingleShard: allocating new open transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:22,504 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-594: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 01:51:22,504 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-594: Message size: 372 bytes, total slices to send: 1 01:51:22,504 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-594: Message does not need to be sliced - sending original message 01:51:22,505 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=3} with Envelope{sessionId=0, txSequence=0, message=ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0}} 01:51:22,505 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=3}, enqueuedTicks=836830534051, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0}} 01:51:22,505 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=3} completed with ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0} 01:51:22,505 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-499438950], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 01:51:22,505 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-499438950], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=836830534051, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 01:51:22,505 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1} 01:51:22,505 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:51:22,505 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 01:51:22,505 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@21ea37f8 01:51:22,505 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} 01:51:22,505 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 2 01:51:22,505 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} sending request ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-499438950], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1239/0x0000000801302f10@678ec3eb 01:51:22,505 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} 01:51:22,505 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@7681136a 01:51:22,505 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#2103460874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:22,505 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6d691e68 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=1}, enqueuedTicks=836836501308} 01:51:22,506 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-595: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=1}} 01:51:22,506 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-595: Message size: 457 bytes, total slices to send: 1 01:51:22,506 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-595: Message does not need to be sliced - sending original message 01:51:22,506 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 3 01:51:22,506 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-499438950], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1236/0x0000000801302878@4c891d98 01:51:22,506 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6d691e68 transmitting entry ConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-499438950], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]}, enqueuedTicks=836836786916} 01:51:22,511 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=1} with Envelope{sessionId=0, txSequence=2, message=ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2}} 01:51:22,511 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=1}, enqueuedTicks=836836501308, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2}} 01:51:22,511 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=1} completed with ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2} 01:51:22,511 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-499438950], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} with Envelope{sessionId=0, txSequence=3, message=ExistsTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, exists=true}} 01:51:22,511 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-499438950], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]}, enqueuedTicks=836836786916, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=ExistsTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, exists=true}} 01:51:22,511 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Exists request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] completed with ExistsTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, exists=true} 01:51:22,511 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4e2d0d47 readied transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 01:51:22,511 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED} allocated sequence 4 01:51:22,511 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@4b3be6f7 01:51:22,511 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#2103460874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:22,511 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6d691e68 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=0, protocol=SIMPLE}, enqueuedTicks=836842236550} 01:51:22,511 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-595: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=0, protocol=SIMPLE}} 01:51:22,511 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-595: Message size: 289 bytes, total slices to send: 1 01:51:22,511 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-595: Message does not need to be sliced - sending original message 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: readying transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 ready 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithSingleShard: transitioned member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 to ready 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct canCommit 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 validated 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct preCommit 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 candidate ready 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1: canCommit - candidate rootPath: / 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1: canCommit - messages: [] 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 prepared 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct commit 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 250 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}]]] 01:51:22,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 submitted to persistence 01:51:22,513 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 01:51:22,513 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}]]] 01:51:22,513 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 01:51:22,513 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:22,513 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=250} 01:51:22,513 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 01:51:22,513 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Log size is now 1 01:51:22,513 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:22,513 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 01:51:22,513 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithSingleShard status sync done true 01:51:22,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: 0, nextIndex: 1 01:51:22,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:51:22,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Setting last applied to 0 01:51:22,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 01:51:22,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 0 data Simple{size=250} 01:51:22,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:22,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@3d4f690a}, purged=MutableUnsignedLongSet{size=0}} 01:51:22,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:51:22,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:22,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1: commit - successfulFromPrevious: [] 01:51:22,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:51:22,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1294849577] with name member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher 01:51:22,514 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1294849577] 01:51:22,514 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:51:22,514 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1294849577] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:22,514 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher: 100.6 μs 01:51:22,515 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:22,515 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 01:51:22,515 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Commit index set to 0 01:51:22,515 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:51:22,515 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Setting last applied to 0 01:51:22,515 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 01:51:22,515 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 0 data Simple{size=250} 01:51:22,515 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:22,516 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4bf24cd1}, purged=MutableUnsignedLongSet{size=0}} 01:51:22,516 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:51:22,516 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:22,516 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#1280266635] with name member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher 01:51:22,516 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#1280266635] 01:51:22,516 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:51:22,516 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 01:51:22,516 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4}} 01:51:22,516 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher: 42.92 μs 01:51:22,516 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=0, protocol=SIMPLE}, enqueuedTicks=836842236550, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4}} 01:51:22,516 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED} directCommit completed 01:51:22,516 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED}: initiating purge 01:51:22,516 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} allocated sequence 5 01:51:22,516 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#1280266635] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:22,516 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-499438950]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@25ed92e 01:51:22,516 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4e2d0d47: creating a new snapshot member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 01:51:22,516 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#2103460874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:22,516 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6d691e68 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-499438950]}, enqueuedTicks=836847138871} 01:51:22,516 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1 for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 01:51:22,516 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:51:22,516 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-499438950], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@5ea46ceb 01:51:22,516 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6d691e68 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-499438950], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=836847318456} 01:51:22,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: purging transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:22,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 01:51:22,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 01:51:22,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:22,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithSingleShard: allocating new open snapshot member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1 01:51:22,517 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-594: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 01:51:22,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 01:51:22,517 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-594: Message size: 395 bytes, total slices to send: 1 01:51:22,517 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-594: Message does not need to be sliced - sending original message 01:51:22,518 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-499438950], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 01:51:22,518 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-499438950], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=836847318456, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 01:51:22,518 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0} 01:51:22,518 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:51:22,519 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:51:22,519 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithSingleShard - waiting on 1 shards 01:51:22,519 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onShutDown 01:51:22,519 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Initiating leader transfer 01:51:22,519 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6842887b 01:51:22,519 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6842887b 01:51:22,519 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Attempting to transfer leadership 01:51:22,519 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:51:22,519 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:51:22,519 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:51:22,519 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:22,519 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}]]] 01:51:22,519 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:22,519 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}]]] 01:51:22,519 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 01:51:22,519 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:51:22,519 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69} 01:51:22,519 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 01:51:22,519 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Log size is now 2 01:51:22,519 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:22,520 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 01:51:22,520 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard] 01:51:22,520 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: 1, nextIndex: 2 01:51:22,520 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:51:22,520 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Setting last applied to 1 01:51:22,520 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 01:51:22,520 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69} 01:51:22,520 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:51:22,520 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: replication of member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@5817f24f 01:51:22,520 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithSingleShard: finished purging transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:22,520 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:22,520 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:51:22,520 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:22,520 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 01:51:22,520 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:22,520 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadWriteTransactionWithSingleShard, matchIndex: 1, lastIndex: 1, isVoting: true 01:51:22,520 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 01:51:22,520 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:22,520 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:22,520 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Leader transfer complete 01:51:22,520 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6d691e68 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@1996c63c, splicing queue 01:51:22,520 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 01:51:22,520 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#2103460874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:22,520 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6d691e68 queue spliced 1 messages 01:51:22,520 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#2103460874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#2103460874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:22,520 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: refreshing backend for shard 1 01:51:22,520 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#2103460874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:22,520 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:22,520 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:22,520 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-28977174] 01:51:22,521 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:22,521 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:51:22,522 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Commit index set to 1 01:51:22,522 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:51:22,522 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Setting last applied to 1 01:51:22,522 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 01:51:22,522 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69} 01:51:22,522 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:51:22,522 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:22,522 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 01:51:22,522 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 01:51:22,522 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5}} not found in inflight queue, checking pending queue 01:51:22,522 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-499438950]} with Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5}} 01:51:22,522 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-499438950]}, enqueuedTicks=836847138871, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5}} 01:51:22,522 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE}: purge completed 01:51:22,522 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@12973f7f purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} 01:51:22,736 PM [cluster-test-shard-dispatcher-28] [WARN] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 213, lastApplied : 1, commitIndex : 1 01:51:22,934 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 01:51:22,934 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#2103460874]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:51:22,934 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 01:51:22,934 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader)_slicer-id-597: Closing 01:51:22,934 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:51:22,934 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithSingleShard 01:51:22,935 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 01:51:22,935 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 01:51:22,935 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 01:51:22,935 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: shutting down 01:51:22,935 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 01:51:22,936 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:22,936 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:22,940 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:22,940 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:22,940 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:22,940 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:22,941 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:51:22,954 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 01:51:22,955 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 01:51:22,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Closing 01:51:22,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:51:22,955 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithSingleShard 01:51:22,955 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:22,955 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:23,952 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [3557717566302733594] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:23,972 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [5198425423885922202] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:23,972 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:23,972 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:23,973 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:23.974] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:23,983 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:23,983 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:23,985 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:23.985] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:23,999 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:24,4 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [1235563391755192132] 01:51:24,4 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:51:24,5 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:51:24,5 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:51:24,5 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:24,6 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:24,17 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:51:24,17 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:51:24,17 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:51:24,20 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:24,24 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-8652280452902903932] 01:51:24,24 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:51:24,25 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:24,25 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:51:24,31 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:24,31 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:24,36 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:24,43 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-381073339840086509] 01:51:24,43 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:51:24,46 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:24,46 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:51:24,53 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:24,54 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:24,54 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:24,54 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:24,55 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSnapshotOnRootOverwrite 01:51:24,58 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#52775176] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:24,60 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-testSnapshotOnRootOverwrite 01:51:24,63 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-testSnapshotOnRootOverwrite 01:51:24,64 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-1-shard-cars-testSnapshotOnRootOverwrite 01:51:24,64 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: no initial generation override, starting from 0 01:51:24,64 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:51:24,64 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1a66313f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@40e0d278 01:51:24,64 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-1-shard-default-testSnapshotOnRootOverwrite 01:51:24,64 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4d6d4ea3 01:51:24,64 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:24,64 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:51:24,64 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:51:24,64 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSnapshotOnRootOverwrite, 0, 1710165084064, None)) 01:51:24,64 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165084063,0,0)) 01:51:24,64 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@41061db1 01:51:24,64 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@40e0d278 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4f663a80 01:51:24,64 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} started 01:51:24,65 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:24,66 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:24,68 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSnapshotOnRootOverwrite 01:51:24,69 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:24,69 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSnapshotOnRootOverwrite 01:51:24,72 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:51:24,72 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:51:24,72 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-cars-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite on shard member-1-shard-cars-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1806594929] 01:51:24,72 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-default-testSnapshotOnRootOverwrite 01:51:24,72 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Shard created : member-1-shard-cars-testSnapshotOnRootOverwrite, persistent : true 01:51:24,73 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-testSnapshotOnRootOverwrite 01:51:24,73 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSnapshotOnRootOverwrite with journal batch size 1 01:51:24,73 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Shard created : member-1-shard-default-testSnapshotOnRootOverwrite, persistent : true 01:51:24,73 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1806594929] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:24,73 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#862370188] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:24,73 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Starting recovery for member-1-shard-default-testSnapshotOnRootOverwrite with journal batch size 1 01:51:24,73 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-notifier#819391693 created and ready for shard:member-1-shard-cars-testSnapshotOnRootOverwrite 01:51:24,73 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-notifier#819391693] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:24,73 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#396202194] 01:51:24,73 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#396202194] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:24,73 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-notifier#1810040525 created and ready for shard:member-1-shard-default-testSnapshotOnRootOverwrite 01:51:24,73 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-570186033] 01:51:24,74 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#1267007396] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:24,74 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:51:24,74 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:51:24,74 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-default-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite on shard member-1-shard-default-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#862370188] 01:51:24,74 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-notifier#1810040525] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:24,74 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-570186033] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:24,76 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:24,76 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:24,76 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:24,76 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:24,76 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [cars] 01:51:24,76 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSnapshotOnRootOverwrite is initialized 01:51:24,76 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [default] 01:51:24,76 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-1-shard-default-testSnapshotOnRootOverwrite is initialized 01:51:24,76 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:51:24,76 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:51:24,76 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 01:51:24,76 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-default-testSnapshotOnRootOverwrite is no longer the leader 01:51:24,76 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from null to Follower 01:51:24,76 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 01:51:24,76 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from null to Follower 01:51:24,76 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 01:51:24,76 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from null to Follower 01:51:24,76 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from null to Follower 01:51:24,80 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: no initial generation override, starting from 0 01:51:24,80 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:51:24,80 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4ae226a4 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@67896f2a 01:51:24,80 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4bb55e17 01:51:24,80 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSnapshotOnRootOverwrite, 0, 1710165084080, None)) 01:51:24,80 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Recovery complete : shard-manager-testSnapshotOnRootOverwrite 01:51:24,80 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-2-shard-cars-testSnapshotOnRootOverwrite 01:51:24,80 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-2-shard-default-testSnapshotOnRootOverwrite 01:51:24,81 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165084079,0,0)) 01:51:24,81 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@67896f2a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@11c0383a 01:51:24,81 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} started 01:51:24,81 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5b207331 01:51:24,81 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:24,81 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSnapshotOnRootOverwrite 01:51:24,82 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-2-shard-default-testSnapshotOnRootOverwrite 01:51:24,83 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Shard created : member-2-shard-cars-testSnapshotOnRootOverwrite, persistent : true 01:51:24,83 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Shard created : member-2-shard-default-testSnapshotOnRootOverwrite, persistent : true 01:51:24,83 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:51:24,83 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-notifier#-1296524101 created and ready for shard:member-2-shard-default-testSnapshotOnRootOverwrite 01:51:24,83 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-1708727672] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:24,83 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#721943804] 01:51:24,83 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite#-1755385511] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:24,83 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-notifier#-1296524101] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:24,83 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#2138249806] 01:51:24,84 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#721943804] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:24,84 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-notifier#-2128426669 created and ready for shard:member-2-shard-cars-testSnapshotOnRootOverwrite 01:51:24,84 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSnapshotOnRootOverwrite with journal batch size 1 01:51:24,84 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Starting recovery for member-2-shard-default-testSnapshotOnRootOverwrite with journal batch size 1 01:51:24,84 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:24,84 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:24,84 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:24,84 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:24,85 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:51:24,85 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-2-shard-default-testSnapshotOnRootOverwrite is no longer the leader 01:51:24,85 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-default-testSnapshotOnRootOverwrite , received role change from null to Follower 01:51:24,85 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [cars] 01:51:24,85 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSnapshotOnRootOverwrite is initialized 01:51:24,85 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [default] 01:51:24,85 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-default-testSnapshotOnRootOverwrite is initialized 01:51:24,85 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-default-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 01:51:24,85 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:24,85 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-2-shard-default-testSnapshotOnRootOverwrite from null to Follower 01:51:24,86 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:24,86 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:51:24,86 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:51:24,86 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:51:24,86 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:51:24,86 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-1-shard-default-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:51:24,86 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:51:24,86 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 01:51:24,86 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSnapshotOnRootOverwrite , received role change from null to Follower 01:51:24,86 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:51:24,86 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 01:51:24,86 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-2-shard-cars-testSnapshotOnRootOverwrite from null to Follower 01:51:24,88 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#2138249806] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:24,88 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-notifier#-2128426669] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:24,95 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:51:24,97 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:24,131 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#52775176] 01:51:24,131 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 01:51:24,131 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1806594929] 01:51:24,297 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:24,297 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Received ElectionTimeout - switching to Candidate 01:51:24,297 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Election: Candidate has following voting peers: [member-2-shard-default-testSnapshotOnRootOverwrite] 01:51:24,297 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=1, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 01:51:24,297 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 1 01:51:24,297 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:24,297 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Closing 01:51:24,297 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:51:24,297 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-default-testSnapshotOnRootOverwrite is no longer the leader 01:51:24,297 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from Follower to Candidate 01:51:24,298 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from Follower to Candidate 01:51:24,307 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:24,307 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Received ElectionTimeout - switching to Candidate 01:51:24,307 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSnapshotOnRootOverwrite] 01:51:24,307 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=1, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 01:51:24,307 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 1 01:51:24,307 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:24,307 PM [cluster-test-shard-dispatcher-13] [DEBUG] MessageAssembler - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Closing 01:51:24,307 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 01:51:24,307 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 01:51:24,307 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from Follower to Candidate 01:51:24,307 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from Follower to Candidate 01:51:24,537 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:51:24,538 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=2, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 01:51:24,538 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:51:24,538 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 2 01:51:24,538 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=2, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 01:51:24,538 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 2 01:51:24,649 PM [cluster-test-shard-dispatcher-18] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:24,688 PM [cluster-test-shard-dispatcher-18] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:24,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:51:24,837 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 01:51:24,837 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 3 01:51:24,847 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:51:24,847 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 01:51:24,847 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 3 01:51:25,18 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:51:25,18 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:51:25,19 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:25,19 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite 01:51:25,19 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:51:25,19 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite 01:51:25,19 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:51:25,19 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:25,19 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:51:25,19 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:51:25,19 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:51:25,19 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:51:25,19 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:51:25,19 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-2-shard-default-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:51:25,31 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:25,31 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite 01:51:25,31 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:51:25,31 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite 01:51:25,31 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:51:25,31 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:25,31 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:51:25,31 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 01:51:25,31 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:51:25,31 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 01:51:25,65 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:51:25,107 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:51:25,107 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 01:51:25,107 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 4 01:51:25,108 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-default-testSnapshotOnRootOverwrite 01:51:25,109 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:25,109 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Cluster state: [] 01:51:25,109 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:25,109 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:25,109 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=null 01:51:25,109 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:25,109 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 01:51:25,109 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:25,110 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:25,110 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:25,110 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Election: Leader has following peers: [member-2-shard-default-testSnapshotOnRootOverwrite] 01:51:25,110 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:25,110 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:25,110 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: true, hasLeader : true 01:51:25,110 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: became leader with frontend state for [] 01:51:25,110 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from Candidate to Leader 01:51:25,111 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 01:51:25,111 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 01:51:25,111 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from Candidate to Leader 01:51:25,111 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 01:51:25,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-default-testSnapshotOnRootOverwrite: Closing 01:51:25,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: getLeaderAddress leaderId = member-1-shard-default-testSnapshotOnRootOverwrite peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 01:51:25,111 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-default-testSnapshotOnRootOverwrite status sync done false 01:51:25,111 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-testSnapshotOnRootOverwrite, leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 01:51:25,111 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 01:51:25,112 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: -1, nextIndex: 0 01:51:25,112 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 01:51:25,112 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:25,167 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 01:51:25,167 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 01:51:25,167 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 4 01:51:25,167 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSnapshotOnRootOverwrite 01:51:25,168 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:25,169 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Cluster state: [] 01:51:25,169 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:25,169 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:25,169 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=null 01:51:25,169 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:25,169 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 01:51:25,169 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:25,170 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:25,170 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:25,170 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Election: Leader has following peers: [member-2-shard-cars-testSnapshotOnRootOverwrite] 01:51:25,170 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:25,170 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:25,170 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: true, hasLeader : true 01:51:25,170 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: became leader with frontend state for [] 01:51:25,170 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from Candidate to Leader 01:51:25,170 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 01:51:25,170 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:25,170 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:25,170 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from Candidate to Leader 01:51:25,170 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All Shards are ready - data store testSnapshotOnRootOverwrite is ready 01:51:25,171 PM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-2-shard-cars-testSnapshotOnRootOverwrite: Closing 01:51:25,171 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: getLeaderAddress leaderId = member-1-shard-cars-testSnapshotOnRootOverwrite peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 01:51:25,171 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-cars-testSnapshotOnRootOverwrite status sync done false 01:51:25,171 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSnapshotOnRootOverwrite, leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 01:51:25,171 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:25,171 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:25,171 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All Shards are ready - data store testSnapshotOnRootOverwrite is ready 01:51:25,172 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: -1, nextIndex: 0 01:51:25,172 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:25,229 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-default-testSnapshotOnRootOverwrite status sync done true 01:51:25,237 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#52775176] 01:51:25,237 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : default 01:51:25,237 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#862370188] 01:51:25,237 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5c3cc029: creating a new transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:51:25,239 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} 01:51:25,239 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:25,239 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:25,239 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#52775176] 01:51:25,239 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:25,239 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} shard 1 01:51:25,239 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1806594929, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@3fe43f15] 01:51:25,239 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:51:25,239 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1806594929, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@3fe43f15] 01:51:25,239 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolving shard 0 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} 01:51:25,239 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 0 to shard default 01:51:25,239 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 0 to shard default 01:51:25,239 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#52775176] 01:51:25,239 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} shard 0 01:51:25,239 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:51:25,239 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:51:25,239 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#862370188, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@78e45fef] 01:51:25,239 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#862370188, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@78e45fef] 01:51:25,240 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=OPEN} allocated sequence 0 01:51:25,240 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:25,240 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5c3cc029 readied transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:51:25,240 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=FLUSHED} sending request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1592138648], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@771ccec4 01:51:25,240 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} scheduling timeout in 30 seconds 01:51:25,240 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1592138648], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@54b986aa 01:51:25,240 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@bf681d1, attempting to connect 01:51:25,240 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:51:25,240 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard default resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@631fe189, attempting to connect 01:51:25,240 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} is not yet known 01:51:25,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} is not yet known 01:51:25,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, nanosAgo=6691, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:51:25,240 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, nanosAgo=15760, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:51:25,240 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:51:25,240 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:51:25,240 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolved shard 0 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#862370188], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present} 01:51:25,241 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#862370188], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 01:51:25,241 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@69e0cac3 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2ce56169 01:51:25,241 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} previous state FLUSHED 01:51:25,241 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2ce56169 01:51:25,241 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:51:25,241 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=OPEN} 01:51:25,241 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1592138648], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=OPEN} 01:51:25,241 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1592138648], modifications=1, protocol=THREE_PHASE} 01:51:25,241 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} allocated sequence 0 01:51:25,241 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1592138648], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@771ccec4 01:51:25,241 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#862370188], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} scheduling timeout in 30 seconds 01:51:25,241 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7bc97490 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1592138648], coordinated=true}, enqueuedTicks=839570871164} 01:51:25,241 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5c3cc029 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#862370188], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 01:51:25,241 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} 01:51:25,241 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@7c20aca9 01:51:25,241 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6e08180c superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@4f5a6b67, splicing queue 01:51:25,241 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6e08180c queue spliced 0 messages 01:51:25,242 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#862370188], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} in 900.5 μs 01:51:25,242 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1806594929], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:51:25,242 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1806594929], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:25,242 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@33eb8406 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@7f87ebe8 01:51:25,242 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:51:25,242 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@7f87ebe8 01:51:25,242 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:51:25,242 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=OPEN} 01:51:25,242 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1592138648], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=OPEN} 01:51:25,242 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1592138648], modifications=1, protocol=THREE_PHASE} 01:51:25,242 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} allocated sequence 0 01:51:25,242 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1592138648], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@54b986aa 01:51:25,242 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1806594929], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:51:25,242 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3440b2d9 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1592138648], coordinated=true}, enqueuedTicks=839570988707} 01:51:25,242 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5c3cc029 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1806594929], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:25,242 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} 01:51:25,242 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@74d8c712 01:51:25,242 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@28953531 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@6f209710, splicing queue 01:51:25,242 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@28953531 queue spliced 0 messages 01:51:25,242 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1806594929], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 667.5 μs 01:51:25,243 PM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-default-testSnapshotOnRootOverwrite: allocating new ready transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:51:25,243 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 initiating canCommit 01:51:25,243 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:51:25,243 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 validated 01:51:25,243 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:25,243 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 completed canCommit 01:51:25,244 PM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSnapshotOnRootOverwrite: allocating new ready transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:51:25,244 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 initiating canCommit 01:51:25,244 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:51:25,244 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 validated 01:51:25,244 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:25,244 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 completed canCommit 01:51:25,244 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1592138648], coordinated=true} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0}} 01:51:25,244 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1592138648], coordinated=true}, enqueuedTicks=839570871164, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0}} 01:51:25,244 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} canCommit completed 01:51:25,244 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1592138648], coordinated=true} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0}} 01:51:25,244 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1592138648], coordinated=true}, enqueuedTicks=839570988707, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0}} 01:51:25,244 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 01:51:25,244 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} allocated sequence 1 01:51:25,244 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@933275d 01:51:25,244 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} 01:51:25,244 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} 01:51:25,244 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} allocated sequence 1 01:51:25,244 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} sending request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@933275d 01:51:25,244 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#862370188], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 01:51:25,244 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7bc97490 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]}, enqueuedTicks=839575206811} 01:51:25,244 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:51:25,244 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@18c8a251 01:51:25,244 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} 01:51:25,244 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 initiating preCommit 01:51:25,244 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} 01:51:25,244 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:51:25,244 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} allocated sequence 1 01:51:25,244 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 candidate ready 01:51:25,244 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@18c8a251 01:51:25,244 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - candidate rootPath: / 01:51:25,244 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1806594929], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:25,245 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3440b2d9 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]}, enqueuedTicks=839575449067} 01:51:25,244 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - messages: [] 01:51:25,245 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 01:51:25,245 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 prepared 01:51:25,245 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:25,245 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 completed preCommit 01:51:25,245 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 initiating preCommit 01:51:25,245 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:51:25,245 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1}} 01:51:25,245 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 candidate ready 01:51:25,245 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1: canCommit - candidate rootPath: / 01:51:25,245 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]}, enqueuedTicks=839575206811, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1}} 01:51:25,245 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:51:25,245 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1: canCommit - messages: [] 01:51:25,245 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 01:51:25,245 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 prepared 01:51:25,245 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:25,245 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 completed preCommit 01:51:25,245 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1}} 01:51:25,245 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]}, enqueuedTicks=839575449067, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1}} 01:51:25,245 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:51:25,246 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} allocated sequence 2 01:51:25,246 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@5350de15 01:51:25,246 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} 01:51:25,246 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} 01:51:25,246 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} allocated sequence 2 01:51:25,246 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} sending request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@5350de15 01:51:25,246 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#862370188], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 01:51:25,246 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7bc97490 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]}, enqueuedTicks=839577024576} 01:51:25,246 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:51:25,246 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@43c902ed 01:51:25,246 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} 01:51:25,246 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} 01:51:25,246 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} allocated sequence 2 01:51:25,246 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@43c902ed 01:51:25,246 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1806594929], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:25,246 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3440b2d9 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]}, enqueuedTicks=839577219251} 01:51:25,246 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 initiating commit 01:51:25,246 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:51:25,246 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 initiating commit 01:51:25,246 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:51:25,246 PM [cluster-test-shard-dispatcher-26] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 113 01:51:25,246 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 01:51:25,247 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 01:51:25,247 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:25,247 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:25,247 PM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 283 01:51:25,247 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 01:51:25,247 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 01:51:25,247 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 01:51:25,247 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}]]] 01:51:25,247 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:25,247 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 submitted to persistence 01:51:25,247 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:25,247 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:51:25,247 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}]]] 01:51:25,247 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 submitted to persistence 01:51:25,248 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 01:51:25,248 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 01:51:25,248 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}]]] 01:51:25,248 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:51:25,248 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:25,248 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=113} 01:51:25,248 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 01:51:25,249 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 1 01:51:25,249 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:25,249 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}]]] 01:51:25,249 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:51:25,249 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:25,249 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=283} 01:51:25,249 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 01:51:25,249 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 1 01:51:25,249 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:25,249 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 0, nextIndex: 1 01:51:25,249 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 0, nextIndex: 1 01:51:25,249 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:51:25,249 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:51:25,249 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 0 01:51:25,249 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 01:51:25,249 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 0 01:51:25,249 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 01:51:25,249 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 0 data Simple{size=283} 01:51:25,249 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 0 data Simple{size=113} 01:51:25,249 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:51:25,249 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:51:25,249 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-1-shard-default-testSnapshotOnRootOverwrite: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@6d7af223}, purged=MutableUnsignedLongSet{size=0}} 01:51:25,249 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-1-shard-cars-testSnapshotOnRootOverwrite: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5a2ab23f}, purged=MutableUnsignedLongSet{size=0}} 01:51:25,249 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-default-testSnapshotOnRootOverwrite: Committed transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:51:25,249 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSnapshotOnRootOverwrite: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} 01:51:25,249 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: commit - successfulFromPrevious: [] 01:51:25,249 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 01:51:25,249 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSnapshotOnRootOverwrite: Committed transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:51:25,250 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1: commit - successfulFromPrevious: [] 01:51:25,250 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:51:25,250 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-default-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-624160102] with name member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher 01:51:25,250 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#1634919878] with name member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#1634919878] 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#1634919878] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2}} 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 64.85 μs 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]}, enqueuedTicks=839577024576, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2}} 01:51:25,250 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-624160102] 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} doCommit completed 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@69e0cac3 completing transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR}: initiating purge 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} allocated sequence 3 01:51:25,250 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@5ec72657 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 24.69 μs 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED}: initiating purge 01:51:25,250 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=DONE} allocated sequence 3 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-624160102] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@6c958aca 01:51:25,250 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] and replicatedToAllIndex is -1 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#862370188], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 01:51:25,250 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=4, lastIndex=0, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7bc97490 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]}, enqueuedTicks=839581082636} 01:51:25,250 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2}} 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]}, enqueuedTicks=839577219251, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2}} 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@33eb8406 completing transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@65b9476 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED}: initiating purge 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=DONE} allocated sequence 3 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@6149ff74 01:51:25,250 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1806594929], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:25,251 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3440b2d9 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]}, enqueuedTicks=839581436295} 01:51:25,251 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[FrontendHistoryMetadata{historyId=0, cookie=0, closed=false, closedTransactions={0=true}, purgedTransactions=ImmutableUnsignedLongSet{size=0}}], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:51:25,251 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: purging transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:51:25,251 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 01:51:25,251 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 01:51:25,251 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:25,251 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:51:25,251 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=4, lastAppliedIndex=0, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@fea3d5f, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:51:25,251 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 0 01:51:25,251 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 0 and term: 4 01:51:25,251 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 01:51:25,251 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 01:51:25,252 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 01:51:25,252 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 01:51:25,252 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5c3cc029: creating a new snapshot member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 01:51:25,252 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-1 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 01:51:25,252 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 01:51:25,252 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:51:25,252 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:25,252 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 01:51:25,252 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: purging transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:51:25,252 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 01:51:25,252 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 01:51:25,253 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:25,253 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] and replicatedToAllIndex is -1 01:51:25,253 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=4, lastIndex=1, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=1, mandatoryTrim=true] 01:51:25,253 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 01:51:25,253 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:51:25,253 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[FrontendHistoryMetadata{historyId=0, cookie=0, closed=false, closedTransactions={0=true}, purgedTransactions=ImmutableUnsignedLongSet{size=0}}], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:51:25,253 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-1-shard-cars-testSnapshotOnRootOverwrite 01:51:25,253 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 01:51:25,253 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-1-shard-default-testSnapshotOnRootOverwrite 01:51:25,253 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onShutDown 01:51:25,253 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating leader transfer 01:51:25,253 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@752d5b2e 01:51:25,253 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@752d5b2e 01:51:25,253 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Attempting to transfer leadership 01:51:25,253 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:25,253 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:51:25,253 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}]]] 01:51:25,253 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:51:25,253 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 01:51:25,253 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=4, lastAppliedIndex=0, lastAppliedTerm=4, unAppliedEntries size=1, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@b06ddb, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:51:25,253 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 0 01:51:25,253 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 0 and term: 4 01:51:25,254 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 01:51:25,254 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 01:51:25,254 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 01:51:25,254 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 01:51:25,254 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a journal response 01:51:25,254 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSnapshotOnRootOverwrite - waiting on 2 shards 01:51:25,254 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 01:51:25,254 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:25,254 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:25,254 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onShutDown 01:51:25,254 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3440b2d9 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@24ae8cb, splicing queue 01:51:25,254 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating leader transfer 01:51:25,254 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1806594929], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:51:25,254 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3440b2d9 queue spliced 1 messages 01:51:25,254 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1806594929], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1806594929], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:25,254 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: refreshing backend for shard 1 01:51:25,254 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1806594929], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:51:25,254 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:25,254 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:25,254 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#52775176] 01:51:25,254 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2e83d157 01:51:25,255 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2e83d157 01:51:25,255 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:25,255 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Attempting to transfer leadership 01:51:25,255 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 01:51:25,255 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:25,255 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 01:51:25,255 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 01:51:25,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 01:51:25,255 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 01:51:25,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 01:51:25,255 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}]]] 01:51:25,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 0 01:51:25,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:51:25,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 0 01:51:25,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 01:51:25,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 0 data Simple{size=283} 01:51:25,255 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 01:51:25,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:51:25,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-testSnapshotOnRootOverwrite: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@3a6e6657}, purged=MutableUnsignedLongSet{size=0}} 01:51:25,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSnapshotOnRootOverwrite: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} 01:51:25,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSnapshotOnRootOverwrite: Committed transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:51:25,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#765977813] with name member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher 01:51:25,256 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:51:25,256 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-cars-testSnapshotOnRootOverwrite status sync done true 01:51:25,256 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 01:51:25,256 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] and replicatedToAllIndex is -1 01:51:25,256 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=4, lastIndex=0, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 01:51:25,256 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 3 01:51:25,256 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[FrontendHistoryMetadata{historyId=0, cookie=0, closed=false, closedTransactions={0=true}, purgedTransactions=ImmutableUnsignedLongSet{size=0}}], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:51:25,256 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 01:51:25,256 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7bc97490 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@749e1f4d, splicing queue 01:51:25,256 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#862370188], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} scheduling timeout in 30 seconds 01:51:25,256 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7bc97490 queue spliced 1 messages 01:51:25,256 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#862370188], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#862370188], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 01:51:25,256 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: refreshing backend for shard 0 01:51:25,256 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#862370188], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present} 01:51:25,256 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Resolving cookie 0 to shard default 01:51:25,256 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractShardBackendResolver - Resolving cookie 0 to shard default 01:51:25,256 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#52775176] 01:51:25,256 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 01:51:25,256 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 01:51:25,257 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#765977813] 01:51:25,257 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:51:25,257 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 01:51:25,257 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSnapshotOnRootOverwrite, matchIndex: 0, lastIndex: 1, isVoting: true 01:51:25,257 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 68.73 μs 01:51:25,257 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#765977813] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:25,257 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:51:25,257 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=4, lastAppliedIndex=0, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@63b19a0a, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:51:25,257 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 0 01:51:25,257 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 0 and term: 4 01:51:25,257 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 01:51:25,257 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 3 01:51:25,257 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 01:51:25,257 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 01:51:25,258 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 01:51:25,259 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 0 01:51:25,259 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:51:25,259 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 0 01:51:25,259 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 01:51:25,259 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 0 data Simple{size=113} 01:51:25,259 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:51:25,259 PM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendMetadata - member-2-shard-default-testSnapshotOnRootOverwrite: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@39a1310}, purged=MutableUnsignedLongSet{size=0}} 01:51:25,259 PM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-default-testSnapshotOnRootOverwrite: Committed transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:51:25,259 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-default-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#1444323633] with name member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher 01:51:25,259 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 01:51:25,259 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}]]] 01:51:25,259 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:51:25,259 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:51:25,259 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57} 01:51:25,259 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 01:51:25,259 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 1 01:51:25,259 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:25,270 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}]]] 01:51:25,270 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 01:51:25,270 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:51:25,270 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56} 01:51:25,270 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 01:51:25,271 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 2 01:51:25,271 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:25,271 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4e71f8e4 aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} 01:51:25,271 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:51:25,271 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#870307478], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@55f1ac76 01:51:25,271 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} 01:51:25,271 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5afe29fd aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 01:51:25,271 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#1444323633] 01:51:25,271 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#870307478]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@1b72c169 01:51:25,271 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 01:51:25,271 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 01:51:25,271 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:51:25,271 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5afe29fd aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-1, state=OPEN} 01:51:25,271 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#870307478]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@470970b 01:51:25,271 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 01:51:25,271 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-809207068], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:51:25,271 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 73.29 μs 01:51:25,271 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@12973f7f aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} 01:51:25,271 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} allocated sequence 1 01:51:25,271 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-499438950], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@55c88d83 01:51:25,271 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#2103460874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:25,271 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#2103460874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:25,271 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#1444323633] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:25,271 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0} 01:51:25,271 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 01:51:25,271 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2ce56169 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0, state=OPEN} 01:51:25,271 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@d820950 01:51:25,271 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#862370188], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 01:51:25,271 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite] 01:51:25,271 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@7f87ebe8 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-1, state=OPEN} 01:51:25,271 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@76e800 01:51:25,271 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1806594929], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:25,271 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 01:51:25,271 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0} 01:51:25,271 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:25,271 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:25,272 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 01:51:25,272 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 01:51:25,272 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite] 01:51:25,272 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] and replicatedToAllIndex is -1 01:51:25,272 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=4, lastIndex=1, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=1, mandatoryTrim=true] 01:51:25,272 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 4 01:51:25,272 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[FrontendHistoryMetadata{historyId=0, cookie=0, closed=false, closedTransactions={0=true}, purgedTransactions=ImmutableUnsignedLongSet{size=0}}], purged=ImmutableUnsignedLongSet{size=0}}]}}} 01:51:25,272 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 01:51:25,272 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 01:51:25,272 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 01:51:25,272 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-default-testSnapshotOnRootOverwrite, matchIndex: 0, lastIndex: 1, isVoting: true 01:51:25,273 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 1, nextIndex: 2 01:51:25,273 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:51:25,273 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 1 01:51:25,273 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 01:51:25,273 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56} 01:51:25,273 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: replication of member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@4eac2e18 01:51:25,273 PM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-1-shard-default-testSnapshotOnRootOverwrite: finished purging transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:51:25,273 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-default-testSnapshotOnRootOverwrite: Purged transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:51:25,273 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3}} not found in inflight queue, checking pending queue 01:51:25,273 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3}} 01:51:25,273 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:25,273 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]}, enqueuedTicks=839581082636, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3}} 01:51:25,273 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=DONE}: purge completed 01:51:25,273 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2ce56169 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=DONE} 01:51:25,273 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR}: purge completed 01:51:25,273 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-default-testSnapshotOnRootOverwrite, matchIndex: 1, lastIndex: 1, isVoting: true 01:51:25,273 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@69e0cac3 purged transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} 01:51:25,273 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Follower's log matches - sending ElectionTimeout 01:51:25,273 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:25,273 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Leader transfer complete 01:51:25,273 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 1, nextIndex: 2 01:51:25,273 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:51:25,273 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 01:51:25,273 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 1 01:51:25,273 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 01:51:25,273 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57} 01:51:25,273 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: replication of member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@5e626049 01:51:25,273 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSnapshotOnRootOverwrite: finished purging transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:51:25,273 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSnapshotOnRootOverwrite: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} 01:51:25,273 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSnapshotOnRootOverwrite: Purged transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:51:25,273 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:25,273 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSnapshotOnRootOverwrite, matchIndex: 1, lastIndex: 1, isVoting: true 01:51:25,273 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Follower's log matches - sending ElectionTimeout 01:51:25,273 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:25,274 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Leader transfer complete 01:51:25,274 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 01:51:25,274 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3}} not found in inflight queue, checking pending queue 01:51:25,274 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3}} 01:51:25,274 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1592138648]}, enqueuedTicks=839581436295, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3}} 01:51:25,274 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 01:51:25,274 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=DONE}: purge completed 01:51:25,274 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@7f87ebe8 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=DONE} 01:51:25,274 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:51:25,274 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@33eb8406 purged transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} 01:51:25,274 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 01:51:25,274 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=4, lastAppliedIndex=0, lastAppliedTerm=4, unAppliedEntries size=1, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@423d4848, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 01:51:25,274 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 0 01:51:25,274 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 0 and term: 4 01:51:25,274 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 4 01:51:25,274 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 4 01:51:25,274 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 01:51:25,274 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 01:51:25,275 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a journal response 01:51:25,280 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 1 01:51:25,280 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:51:25,280 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 1 01:51:25,280 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 01:51:25,280 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56} 01:51:25,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-default-testSnapshotOnRootOverwrite: Purged transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 01:51:25,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 01:51:25,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 1 01:51:25,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:51:25,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 1 01:51:25,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 01:51:25,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57} 01:51:25,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSnapshotOnRootOverwrite: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} 01:51:25,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSnapshotOnRootOverwrite: Purged transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 01:51:25,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 01:51:25,667 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All shards shutdown - sending PoisonPill to self 01:51:25,667 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-testSnapshotOnRootOverwrite: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1806594929]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:51:25,668 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Stopping Shard member-1-shard-default-testSnapshotOnRootOverwrite 01:51:25,668 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-default-testSnapshotOnRootOverwrite (Leader)_slicer-id-604: Closing 01:51:25,668 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 01:51:25,668 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSnapshotOnRootOverwrite 01:51:25,668 PM [cluster-test-shard-dispatcher-13] [DEBUG] MessageSlicer - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader)_slicer-id-605: Closing 01:51:25,668 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 01:51:25,669 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-testSnapshotOnRootOverwrite 01:51:25,669 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 01:51:25,669 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 01:51:25,669 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager$ShardNotInitializedTimeout] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#52775176] to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#52775176] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#52775176] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:25,669 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: shutting down 01:51:25,669 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 01:51:25,669 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 01:51:25,670 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:25,670 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:25,674 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:25,674 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:25,674 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:25,674 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:25,676 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:51:25,688 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSnapshotOnRootOverwrite: Closing 01:51:25,689 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSnapshotOnRootOverwrite 01:51:25,689 PM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Closing 01:51:25,689 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 01:51:25,689 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Stopping Shard member-2-shard-default-testSnapshotOnRootOverwrite 01:51:25,689 PM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Closing 01:51:25,689 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 01:51:25,689 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-testSnapshotOnRootOverwrite 01:51:25,690 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:25,690 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:26,685 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [1235563391755192132] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:26,706 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-8652280452902903932] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:26,711 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:26,711 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:26,712 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:26.713] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:26,720 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:26,720 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:26,722 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:26.723] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:26,744 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:26,749 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [7821753892465524619] 01:51:26,749 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:51:26,751 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:51:26,751 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:51:26,751 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:26,751 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:26,761 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:51:26,761 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:51:26,761 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:51:26,763 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:26,767 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-8750557346820639847] 01:51:26,768 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:51:26,768 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:26,768 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:51:26,771 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:26,771 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:26,789 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:26,796 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-466222286183353458] 01:51:26,796 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:51:26,798 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:26,798 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:51:26,798 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:26,798 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:26,808 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:26,809 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:26,810 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 01:51:26,820 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 01:51:26,820 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-1790060182] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:26,822 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: no initial generation override, starting from 0 01:51:26,823 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:51:26,823 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1ede95d9 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1703766c 01:51:26,823 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2c50810b 01:51:26,823 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 01:51:26,823 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-1-shard-cars-testWriteTransactionWithSingleShard 01:51:26,823 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:26,823 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:51:26,823 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1710165086823, None)) 01:51:26,823 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165086822,0,0)) 01:51:26,823 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@44048539 01:51:26,824 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1703766c to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@47746d11 01:51:26,824 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} started 01:51:26,824 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:26,825 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:26,825 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testWriteTransactionWithSingleShard 01:51:26,826 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:26,827 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 01:51:26,840 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:51:26,849 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:26,863 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:51:26,865 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:26,866 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:51:26,866 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:51:26,866 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard on shard member-1-shard-cars-testWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231] 01:51:26,866 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Shard created : member-1-shard-cars-testWriteTransactionWithSingleShard, persistent : true 01:51:26,867 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Starting recovery for member-1-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 01:51:26,867 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:26,867 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-member-1:snapshot-read#888317305] 01:51:26,867 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-notifier#-1205632677 created and ready for shard:member-1-shard-cars-testWriteTransactionWithSingleShard 01:51:26,867 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-member-1:snapshot-read#888317305] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:26,867 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-notifier#-1205632677] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:26,867 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:26,867 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:26,868 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:51:26,868 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 01:51:26,868 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 01:51:26,868 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-cars-testWriteTransactionWithSingleShard is initialized 01:51:26,868 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 01:51:26,868 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard 01:51:26,868 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from null to Follower 01:51:26,870 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#1308239829] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:26,871 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 01:51:26,872 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: no initial generation override, starting from 0 01:51:26,872 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:51:26,872 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@48c9e1f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@dcb2d6d 01:51:26,872 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 01:51:26,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testWriteTransactionWithSingleShard 01:51:26,872 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:26,872 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:51:26,873 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@454ccbfd 01:51:26,873 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1710165086873, None)) 01:51:26,873 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165086872,0,0)) 01:51:26,873 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@32624d20 01:51:26,873 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@dcb2d6d to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3446c6b2 01:51:26,873 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} started 01:51:26,873 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:26,873 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithSingleShard 01:51:26,874 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:51:26,874 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:51:26,874 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard on shard member-2-shard-cars-testWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-68737575] 01:51:26,874 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithSingleShard, persistent : true 01:51:26,874 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 01:51:26,874 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#1059266721 created and ready for shard:member-2-shard-cars-testWriteTransactionWithSingleShard 01:51:26,874 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#1528121111] 01:51:26,874 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-68737575] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:26,875 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#1059266721] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:26,875 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#1528121111] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:26,875 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:26,875 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:26,875 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 01:51:26,875 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithSingleShard is initialized 01:51:26,875 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:51:26,875 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 01:51:26,875 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 01:51:26,875 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-1-shard-cars-testWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:51:26,875 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard 01:51:26,875 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testWriteTransactionWithSingleShard from null to Follower 01:51:26,924 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-1790060182] 01:51:26,924 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 01:51:26,924 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231] 01:51:27,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:27,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 01:51:27,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testWriteTransactionWithSingleShard] 01:51:27,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:51:27,180 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 1 01:51:27,180 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:27,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 01:51:27,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:51:27,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 01:51:27,180 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 01:51:27,180 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from Follower to Candidate 01:51:27,430 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:51:27,430 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:51:27,430 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 2 01:51:27,491 PM [cluster-test-shard-dispatcher-25] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:27,710 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:51:27,710 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:51:27,710 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 3 01:51:27,771 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:51:27,771 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:51:27,771 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:27,771 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard 01:51:27,771 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:51:27,771 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:27,771 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:51:27,771 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:51:27,771 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Peer address for peer member-2-shard-cars-testWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:51:27,777 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:27,777 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard 01:51:27,777 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:51:27,777 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:27,777 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:51:27,777 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 01:51:27,809 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:51:28,30 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:51:28,30 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:51:28,30 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 4 01:51:28,30 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testWriteTransactionWithSingleShard 01:51:28,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:28,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Cluster state: [] 01:51:28,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:28,32 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:28,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 01:51:28,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:28,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:51:28,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:28,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:28,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:28,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testWriteTransactionWithSingleShard] 01:51:28,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:28,34 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:28,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:51:28,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: became leader with frontend state for [] 01:51:28,34 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 01:51:28,34 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:51:28,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:28,34 PM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:28,34 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from Candidate to Leader 01:51:28,34 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 01:51:28,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard: Closing 01:51:28,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testWriteTransactionWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 01:51:28,35 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:51:28,35 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:28,35 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:28,35 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 01:51:28,35 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithSingleShard status sync done false 01:51:28,36 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: -1, nextIndex: 0 01:51:28,36 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:28,83 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@22678b9c: creating a new transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:51:28,83 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} 01:51:28,83 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:28,83 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:28,83 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#1308239829] 01:51:28,83 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} shard 1 01:51:28,83 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:51:28,83 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:28,84 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@22678b9c readied transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:51:28,84 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=4, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@3bb0c48f 01:51:28,84 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:51:28,84 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:28,84 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard, primaryVersion=13] 01:51:28,84 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard, primaryVersion=13] 01:51:28,84 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@2b8f2afe, attempting to connect 01:51:28,85 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} is not yet known 01:51:28,86 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, nanosAgo=13330, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:51:28,87 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:28,87 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:28,87 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:28,87 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7b54709b instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@388db054 01:51:28,87 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:51:28,87 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@388db054 01:51:28,87 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:28,87 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 01:51:28,87 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=4, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 01:51:28,87 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:28,88 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=4, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2277/0x00000008016c5c10@2955ea05 01:51:28,88 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:28,88 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7c8ff06b transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=4, protocol=SIMPLE}, enqueuedTicks=842414648690} 01:51:28,88 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-608: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=4, protocol=SIMPLE}} 01:51:28,88 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-608: Message size: 509 bytes, total slices to send: 1 01:51:28,88 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-608: Message does not need to be sliced - sending original message 01:51:28,88 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@22678b9c finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:28,88 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 01:51:28,88 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@2e805259 01:51:28,88 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@67e76c90 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@2f74daed, splicing queue 01:51:28,88 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@67e76c90 queue spliced 0 messages 01:51:28,89 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.762 ms 01:51:28,92 PM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:28,92 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: readying transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:28,92 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 ready 01:51:28,92 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: transitioned member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 to ready 01:51:28,92 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct canCommit 01:51:28,92 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:28,92 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 validated 01:51:28,92 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:28,92 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct preCommit 01:51:28,92 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:28,92 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 candidate ready 01:51:28,92 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1: canCommit - candidate rootPath: / 01:51:28,92 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1: canCommit - messages: [] 01:51:28,92 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 01:51:28,92 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 prepared 01:51:28,92 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:28,92 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct commit 01:51:28,92 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:28,93 PM [cluster-test-shard-dispatcher-15] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 246 01:51:28,93 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 01:51:28,93 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 01:51:28,93 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:28,93 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:28,93 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 01:51:28,93 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}]]] 01:51:28,93 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 submitted to persistence 01:51:28,94 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 01:51:28,94 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}]]] 01:51:28,94 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 01:51:28,94 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:28,94 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=246} 01:51:28,94 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 01:51:28,94 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 1 01:51:28,94 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:28,95 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 01:51:28,95 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithSingleShard status sync done true 01:51:28,95 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 0, nextIndex: 1 01:51:28,95 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:51:28,95 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 0 01:51:28,95 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 01:51:28,95 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 0 data Simple{size=246} 01:51:28,95 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:28,95 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendMetadata - member-1-shard-cars-testWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@61b0c9e2}, purged=MutableUnsignedLongSet{size=0}} 01:51:28,95 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:51:28,95 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:28,95 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1: commit - successfulFromPrevious: [] 01:51:28,95 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:51:28,95 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#468649612] with name member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher 01:51:28,96 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#468649612] 01:51:28,96 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:51:28,96 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 88.09 μs 01:51:28,96 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:28,96 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 01:51:28,96 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#468649612] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:28,97 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 0 01:51:28,97 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:51:28,97 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 0 01:51:28,97 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 01:51:28,97 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 0 data Simple{size=246} 01:51:28,98 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:28,98 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@759fb93c}, purged=MutableUnsignedLongSet{size=0}} 01:51:28,98 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:51:28,98 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:28,98 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-1496739843] with name member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher 01:51:28,98 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=4, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0}} 01:51:28,98 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-1496739843] 01:51:28,98 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=4, protocol=SIMPLE}, enqueuedTicks=842414648690, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0}} 01:51:28,98 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=4, protocol=SIMPLE} completed with TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0} 01:51:28,98 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} directCommit completed 01:51:28,98 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 01:51:28,98 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:51:28,98 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:51:28,98 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1868627509]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@12c52aa3 01:51:28,98 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} 01:51:28,98 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SEALED}: initiating purge 01:51:28,98 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} allocated sequence 1 01:51:28,98 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1868627509]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@3efbb038 01:51:28,98 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-1496739843] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:28,98 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:28,98 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7c8ff06b transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1868627509]}, enqueuedTicks=842429323472} 01:51:28,99 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@22678b9c: creating a new snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 01:51:28,99 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 01:51:28,99 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:51:28,99 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@50669d8d 01:51:28,99 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7c8ff06b transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=842429728742} 01:51:28,99 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 94.34 μs 01:51:28,99 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 01:51:28,100 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: purging transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:28,100 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 01:51:28,101 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 01:51:28,101 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:28,101 PM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1 01:51:28,101 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-607: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 01:51:28,101 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-607: Message size: 391 bytes, total slices to send: 1 01:51:28,101 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-607: Message does not need to be sliced - sending original message 01:51:28,101 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 01:51:28,106 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 01:51:28,106 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=842429728742, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 01:51:28,106 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0} 01:51:28,106 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5760ad86: creating a new snapshot member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:51:28,106 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} 01:51:28,106 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:28,106 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:28,106 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-1790060182] 01:51:28,106 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:28,106 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} shard 1 01:51:28,106 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@3a71402c] 01:51:28,106 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 01:51:28,106 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:28,106 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-43977299], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@2ec3822 01:51:28,106 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@3a71402c] 01:51:28,106 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:51:28,107 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@e1b8252, attempting to connect 01:51:28,107 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} is not yet known 01:51:28,107 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, nanosAgo=6410, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:51:28,107 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:51:28,107 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:51:28,107 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:28,107 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@608d22bc instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@c253bdd 01:51:28,107 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:51:28,108 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@c253bdd 01:51:28,108 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:28,108 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 01:51:28,108 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-43977299], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 01:51:28,108 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5760ad86 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:28,108 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 01:51:28,108 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@1186dcab 01:51:28,108 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2d35fef9 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@86bc351, splicing queue 01:51:28,108 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2d35fef9 queue spliced 0 messages 01:51:28,108 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 559.8 μs 01:51:28,108 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0} 01:51:28,110 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@22678b9c: creating a new transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:51:28,110 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:51:28,110 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=OPEN} allocated sequence 0 01:51:28,110 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@22678b9c readied transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 01:51:28,111 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@326247ee 01:51:28,111 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7c8ff06b transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=1, protocol=SIMPLE}, enqueuedTicks=842441598345} 01:51:28,111 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-608: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, message: Envelope{sessionId=0, txSequence=3, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=1, protocol=SIMPLE}} 01:51:28,111 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-608: Message size: 417 bytes, total slices to send: 1 01:51:28,111 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-608: Message does not need to be sliced - sending original message 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: readying transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 ready 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: transitioned member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 to ready 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 initiating direct canCommit 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 validated 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 initiating direct preCommit 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 candidate ready 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1: canCommit - candidate rootPath: / 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1: canCommit - messages: [] 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1: preCommit - successfulFromPrevious: [] 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 prepared 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 initiating direct commit 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 213 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 01:51:28,112 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:28,113 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:51:28,113 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 01:51:28,113 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}]]] 01:51:28,113 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 submitted to persistence 01:51:28,113 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 01:51:28,114 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}]]] 01:51:28,114 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 2 01:51:28,114 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:51:28,114 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65} 01:51:28,114 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 01:51:28,114 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=213} 01:51:28,114 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 01:51:28,114 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 3 01:51:28,114 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:28,114 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 01:51:28,114 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 01:51:28,115 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 2, nextIndex: 3 01:51:28,115 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 01:51:28,115 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 1 01:51:28,115 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 01:51:28,115 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65} 01:51:28,115 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: replication of member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@1e59ff6d 01:51:28,115 PM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: finished purging transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:28,115 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:51:28,115 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:28,115 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 2 01:51:28,115 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 01:51:28,115 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 2 data Simple{size=213} 01:51:28,115 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:51:28,115 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:51:28,115 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:51:28,115 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1: commit - successfulFromPrevious: [] 01:51:28,115 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 changing state from COMMIT_PENDING to COMMITTED 01:51:28,115 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:51:28,116 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 70.46 μs 01:51:28,116 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 1 for term 4 01:51:28,116 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:51:28,116 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=2 01:51:28,116 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1868627509]} with Envelope{sessionId=0, txSequence=1, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 01:51:28,117 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1868627509]}, enqueuedTicks=842429323472, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 01:51:28,117 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE}: purge completed 01:51:28,117 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@388db054 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} 01:51:28,117 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:51:28,117 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7b54709b purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 01:51:28,117 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=1, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0}} 01:51:28,117 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=1, protocol=SIMPLE}, enqueuedTicks=842441598345, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0}} 01:51:28,117 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=FLUSHED} directCommit completed 01:51:28,117 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=FLUSHED}: initiating purge 01:51:28,117 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=DONE} allocated sequence 1 01:51:28,117 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1868627509]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@cb367b1 01:51:28,117 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:28,117 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7c8ff06b transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1868627509]}, enqueuedTicks=842447809779} 01:51:28,117 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@22678b9c: creating a new snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 01:51:28,117 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 01:51:28,117 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} allocated sequence 0 01:51:28,117 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} sending request ExistsTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1239/0x0000000801302f10@3634c579 01:51:28,117 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7c8ff06b transmitting entry ConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]}, enqueuedTicks=842448359702} 01:51:28,117 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 2 01:51:28,117 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 01:51:28,117 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 1 01:51:28,117 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 01:51:28,117 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65} 01:51:28,117 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:51:28,118 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:28,118 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 2 01:51:28,118 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 01:51:28,118 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 2 data Simple{size=213} 01:51:28,118 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:51:28,118 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: purging transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:51:28,118 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}] 01:51:28,118 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}] 01:51:28,118 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:51:28,118 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:28,118 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:51:28,118 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 1 for term 4 01:51:28,118 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=2 01:51:28,118 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 01:51:28,118 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}]]] 01:51:28,118 PM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1 01:51:28,118 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}] 01:51:28,118 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 01:51:28,119 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 41.85 μs 01:51:28,119 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ExistsTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} with Envelope{sessionId=0, txSequence=5, message=ExistsTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, exists=true}} 01:51:28,119 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]}, enqueuedTicks=842448359702, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=ExistsTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, exists=true}} 01:51:28,119 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Exists request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] completed with ExistsTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, exists=true} 01:51:28,122 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@22678b9c: creating a new snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 01:51:28,122 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 01:51:28,122 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} allocated sequence 0 01:51:28,122 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@62761af7 01:51:28,122 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7c8ff06b transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=842453351625} 01:51:28,123 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}]]] 01:51:28,123 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 01:51:28,123 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:51:28,123 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66} 01:51:28,123 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}] 01:51:28,123 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 2 01:51:28,123 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:28,123 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}] 01:51:28,123 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 3, nextIndex: 4 01:51:28,123 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 01:51:28,123 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 3 01:51:28,123 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 2 to 3 01:51:28,123 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66} 01:51:28,123 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: replication of member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@47bcb509 01:51:28,123 PM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: finished purging transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:51:28,123 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:51:28,124 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:51:28,124 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 2 for term 4 01:51:28,124 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:51:28,124 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=3 01:51:28,124 PM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1 01:51:28,124 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-607: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, message: Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0}} 01:51:28,124 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-607: Message size: 369 bytes, total slices to send: 1 01:51:28,124 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-607: Message does not need to be sliced - sending original message 01:51:28,124 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 3 01:51:28,124 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 01:51:28,124 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1868627509]} with Envelope{sessionId=0, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=1}} 01:51:28,124 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 3 01:51:28,124 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 2 to 3 01:51:28,124 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1868627509]}, enqueuedTicks=842447809779, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=1}} 01:51:28,124 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=DONE}: purge completed 01:51:28,124 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@388db054 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=DONE} 01:51:28,124 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66} 01:51:28,124 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:51:28,124 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:51:28,124 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 2 for term 4 01:51:28,124 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=3 01:51:28,125 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0}} 01:51:28,125 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1868627509], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=842453351625, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0}} 01:51:28,125 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0} 01:51:28,125 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5760ad86: creating a new snapshot member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 01:51:28,126 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1 for transaction member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 01:51:28,126 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-1790060182] 01:51:28,126 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 01:51:28,127 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231] 01:51:28,127 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231] 01:51:28,178 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:51:28,178 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Stopping Shard member-1-shard-cars-testWriteTransactionWithSingleShard 01:51:28,178 PM [cluster-test-shard-dispatcher-15] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader)_slicer-id-610: Closing 01:51:28,178 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:51:28,179 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 01:51:28,179 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:28,179 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:28,188 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:28,188 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:28,188 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:28,188 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:28,190 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:51:28,201 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:51:28,202 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithSingleShard 01:51:28,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 01:51:28,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:51:28,202 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 01:51:28,202 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:28,202 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:29,199 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [7821753892465524619] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:29,219 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-8750557346820639847] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:29,223 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:29,223 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:29,224 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:29.226] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:29,248 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:51:29,254 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://reinstated-member2@127.0.0.1:2559] with UID [3166106260606049535] 01:51:29,254 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:51:29,255 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Registered cluster JMX MBean [akka:type=Cluster] 01:51:29,255 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Started up successfully 01:51:29,255 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:29,255 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:29,263 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:29,264 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:29,265 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 01:51:29,271 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2108112471] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:29,272 PM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 01:51:29,273 PM [reinstated-member2-akka.actor.default-dispatcher-18] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: recovered identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:51:29,273 PM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 01:51:29,273 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testWriteTransactionWithSingleShard 01:51:29,273 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 01:51:29,273 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1160a57e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@544b0d29 01:51:29,273 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-people-testWriteTransactionWithSingleShard 01:51:29,273 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4330821a 01:51:29,273 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1710165089273, None)) 01:51:29,274 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165089272,0,0)) 01:51:29,274 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@65c9fef2 01:51:29,274 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@544b0d29 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3cac5ace 01:51:29,274 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} started 01:51:29,274 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:29,274 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithSingleShard 01:51:29,274 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testWriteTransactionWithSingleShard 01:51:29,275 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithSingleShard, persistent : true 01:51:29,275 PM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#210029802] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:29,275 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#924142818 created and ready for shard:member-2-shard-cars-testWriteTransactionWithSingleShard 01:51:29,275 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-1489256933] 01:51:29,275 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#924142818] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:29,275 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 01:51:29,275 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-1489256933] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:29,275 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Shard created : member-2-shard-people-testWriteTransactionWithSingleShard, persistent : true 01:51:29,275 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard#-1028492963] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:29,275 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard-notifier#1763417849 created and ready for shard:member-2-shard-people-testWriteTransactionWithSingleShard 01:51:29,275 PM [reinstated-member2-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#288882079] 01:51:29,276 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard-notifier#1763417849] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:29,276 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-2-shard-people-testWriteTransactionWithSingleShard with journal batch size 1 01:51:29,276 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#288882079] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://reinstated-member2/deadLetters] 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://reinstated-member2/deadLetters] 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 0, size: 246 01:51:29,277 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://reinstated-member2/deadLetters] 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 0 to 0 01:51:29,277 PM [reinstated-member2-shard-dispatcher-13] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 01:51:29,277 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:51:29,277 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithSingleShard is no longer the leader 01:51:29,277 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from null to Follower 01:51:29,277 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Follower): Received TimeoutNow - switching to Candidate 01:51:29,277 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [] 01:51:29,277 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-2-shard-people-testWriteTransactionWithSingleShard 01:51:29,277 PM [reinstated-member2-shard-dispatcher-13] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Starting new election term 1 01:51:29,277 PM [reinstated-member2-shard-dispatcher-13] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:29,277 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithSingleShard (Follower): Closing 01:51:29,277 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:51:29,277 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithSingleShard is no longer the leader 01:51:29,277 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [people] 01:51:29,277 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-people-testWriteTransactionWithSingleShard is initialized 01:51:29,277 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1f0bf85}, purged=MutableUnsignedLongSet{size=0}} 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 1, size: 65 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 2, size: 213 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 1 to 2 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 01:51:29,277 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 2 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 3, size: 66 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 3 to 3 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 2 to 3 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://reinstated-member2/deadLetters] 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Recovery completed in 1.195 ms - Switching actor to Follower - last log index = 3, last log term = 4, snapshot index = -1, snapshot term = -1, journal size = 4 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:51:29,278 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 01:51:29,278 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithSingleShard is initialized 01:51:29,278 PM [reinstated-member2-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Received TimeoutNow - switching to Candidate 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [] 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=5, votedFor=member-2-shard-cars-testWriteTransactionWithSingleShard 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 5 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:51:29,278 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 01:51:29,279 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 01:51:29,279 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:51:29,279 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 01:51:29,279 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [] 01:51:29,279 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [] 01:51:29,279 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 01:51:29,279 PM [reinstated-member2-shard-dispatcher-12] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:51:29,279 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:51:29,279 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:51:29,279 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: became leader with frontend state for [] 01:51:29,279 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 01:51:29,279 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , registered listener akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard 01:51:29,279 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: became leader with frontend state for [member-2-frontend-datastore-testWriteTransactionWithSingleShard] 01:51:29,279 PM [reinstated-member2-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithSingleShard, leaderId=member-2-shard-people-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:51:29,279 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 01:51:29,279 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:29,279 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:29,279 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard 01:51:29,279 PM [reinstated-member2-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-people-testWriteTransactionWithSingleShard from Candidate to Leader 01:51:29,279 PM [reinstated-member2-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 01:51:29,279 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:29,279 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:29,279 PM [reinstated-member2-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testWriteTransactionWithSingleShard from Candidate to Leader 01:51:29,279 PM [reinstated-member2-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 01:51:29,324 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2108112471] 01:51:29,324 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 01:51:29,324 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#210029802] 01:51:29,325 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@53c2657a: creating a new snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 01:51:29,325 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} 01:51:29,325 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:29,325 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:29,325 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2108112471] 01:51:29,325 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, history=0, cookie=0} shard 1 01:51:29,325 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:29,325 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 01:51:29,325 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} allocated sequence 0 01:51:29,325 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#210029802, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@256424a5] 01:51:29,325 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=0, replyTo=Actor[akka://reinstated-member2/user/$a#-756160416], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@132fa9e7 01:51:29,325 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} scheduling timeout in 30 seconds 01:51:29,327 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#210029802, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@256424a5] 01:51:29,327 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@f5d53d, attempting to connect 01:51:29,333 PM [reinstated-member2-shard-dispatcher-15] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: retiring state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, nanosAgo=54231849, purgedHistories=MutableUnsignedLongSet{size=0}}, outdated by request from client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 01:51:29,333 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, nanosAgo=7961, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 01:51:29,334 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:51:29,334 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#210029802], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:51:29,334 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#210029802], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:29,334 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@50e50a96 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@780b5c28 01:51:29,334 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} previous state OPEN 01:51:29,334 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@780b5c28 01:51:29,334 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1 01:51:29,334 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} 01:51:29,334 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=0, replyTo=Actor[akka://reinstated-member2/user/$a#-756160416], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} 01:51:29,335 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@53c2657a finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#210029802], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:29,335 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} 01:51:29,335 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@7f0aa2a3 01:51:29,335 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5571a2b8 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@77394e22, splicing queue 01:51:29,335 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5571a2b8 queue spliced 0 messages 01:51:29,335 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#210029802], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 496.5 μs 01:51:29,335 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=0} 01:51:29,335 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 01:51:29,335 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-cars-testWriteTransactionWithSingleShard 01:51:29,335 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-people-testWriteTransactionWithSingleShard 01:51:29,335 PM [reinstated-member2-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithSingleShard - waiting on 2 shards 01:51:29,335 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onShutDown 01:51:29,335 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@698c7706 01:51:29,335 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onShutDown 01:51:29,335 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@1e381879 01:51:29,335 PM [reinstated-member2-shard-dispatcher-15] [DEBUG] ShardDataTree - member-2-shard-people-testWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@1e381879 01:51:29,335 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@698c7706 01:51:29,336 PM [reinstated-member2-shard-dispatcher-12] [INFO] TestShard - Stopping Shard member-2-shard-people-testWriteTransactionWithSingleShard 01:51:29,336 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] MessageSlicer - member-2-shard-people-testWriteTransactionWithSingleShard (Leader)_slicer-id-614: Closing 01:51:29,336 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithSingleShard 01:51:29,336 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] MessageSlicer - member-2-shard-cars-testWriteTransactionWithSingleShard (Leader)_slicer-id-615: Closing 01:51:29,336 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:51:29,336 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 01:51:29,336 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 01:51:29,336 PM [reinstated-member2-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 01:51:29,336 PM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:51:29,336 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:51:29,336 PM [reinstated-member2-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: shutting down 01:51:29,336 PM [reinstated-member2-akka.actor.default-dispatcher-20] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:51:29,336 PM [main] [WARN] ActorUtils - ShardManager for testWriteTransactionWithSingleShard data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:51:29,337 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:51:29,337 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 01:51:29,337 PM [main] [WARN] ActorUtils - ShardManager for testWriteTransactionWithSingleShard data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:51:29,337 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://reinstated-member2/user/$a#-756160416] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/$a#-756160416] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:29,337 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 01:51:29,337 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2108112471] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2108112471] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:29,337 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2108112471] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-2108112471] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:29,338 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:29,338 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:29,340 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:51:29,354 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:29,354 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:29,354 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:29.355] [reinstated-member2-akka.actor.internal-dispatcher-4] [Cluster(akka://reinstated-member2)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:29,364 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:29,372 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [1567168438696134565] 01:51:29,372 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:51:29,373 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:51:29,373 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:51:29,374 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:29,374 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:29,389 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:51:29,389 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:51:29,389 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:51:29,389 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:51:29,394 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-6118546351265688535] 01:51:29,394 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:51:29,396 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:29,396 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:51:29,405 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:29,405 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:29,409 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:51:29,416 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [124135859621267304] 01:51:29,416 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:51:29,422 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:29,422 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:51:29,422 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:29,422 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:29,429 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:29,431 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:29,432 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,443 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: no initial generation override, starting from 0 01:51:29,444 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:51:29,444 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@550da764 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@39318bee 01:51:29,444 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7c83eb08 01:51:29,444 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, 0, 1710165089444, None)) 01:51:29,444 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165089443,0,0)) 01:51:29,444 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@39318bee to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6787264d 01:51:29,444 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} started 01:51:29,444 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-816717275] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:29,445 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,445 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:29,447 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:29,448 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,449 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,449 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Creating local shard: member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,449 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1aa5d283 01:51:29,450 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:29,450 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,450 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:29,450 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,450 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,451 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, persistent : true 01:51:29,451 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with journal batch size 1 01:51:29,451 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:29,451 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#522483573] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:29,452 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-1444889622] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:29,452 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#522483573 created and ready for shard:member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,453 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,453 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:29,453 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:29,454 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:51:29,454 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 01:51:29,454 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Initializing shard [cars] 01:51:29,454 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is initialized 01:51:29,456 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:51:29,457 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-1:snapshot-read#-2078559999] 01:51:29,457 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-1:snapshot-read#-2078559999] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:29,457 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:51:29,457 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from null to Follower 01:51:29,458 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,458 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from null to Follower 01:51:29,459 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:29,460 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:29,460 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: no initial generation override, starting from 0 01:51:29,460 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:51:29,460 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@22f2145 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@79950bfe 01:51:29,461 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Creating local shard: member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,461 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@17e24c1 01:51:29,461 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:29,461 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,466 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, 0, 1710165089466, None)) 01:51:29,467 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165089465,0,0)) 01:51:29,467 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@15ecc43d 01:51:29,467 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@79950bfe to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7bea3 01:51:29,467 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} started 01:51:29,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:29,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,468 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: updated peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1167697833] 01:51:29,468 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, persistent : true 01:51:29,468 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1167697833] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:29,468 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#534113158 created and ready for shard:member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,468 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with journal batch size 1 01:51:29,469 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#534113158] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:29,469 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:29,469 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:29,469 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Initializing shard [cars] 01:51:29,469 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is initialized 01:51:29,469 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:51:29,469 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 01:51:29,469 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,469 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from null to Follower 01:51:29,469 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,469 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from null to Follower 01:51:29,474 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-2:snapshot-read#-2133303833] 01:51:29,474 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-2:snapshot-read#-2133303833] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:29,517 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-816717275] 01:51:29,518 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: findLocalShard : cars 01:51:29,518 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261] 01:51:29,671 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:29,671 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Received ElectionTimeout - switching to Candidate 01:51:29,671 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 01:51:29,671 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,672 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 1 01:51:29,672 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:29,672 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 01:51:29,672 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:51:29,672 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 01:51:29,672 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Follower to Candidate 01:51:29,672 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Follower to Candidate 01:51:29,921 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:51:29,921 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:29,921 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 2 01:51:30,66 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:30,171 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:51:30,171 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:30,172 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 3 01:51:30,392 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:51:30,392 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:51:30,393 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:30,393 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:30,393 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:30,393 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:30,393 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:30,393 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:30,393 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:30,401 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:51:30,401 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:30,401 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 4 01:51:30,402 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:30,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:30,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Cluster state: [] 01:51:30,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:30,403 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:30,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=null 01:51:30,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:30,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:30,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:30,408 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:30,408 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:30,408 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 01:51:30,408 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:30,408 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:30,408 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:51:30,408 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: became leader with frontend state for [] 01:51:30,411 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Candidate to Leader 01:51:30,411 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderPayloadVersion=13] 01:51:30,411 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:30,411 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:30,411 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Candidate to Leader 01:51:30,411 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All Shards are ready - data store testTransactionWithCreateTxFailureDueToNoLeader is ready 01:51:30,412 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:51:30,412 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:30,413 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader status sync done false 01:51:30,413 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderPayloadVersion=13] 01:51:30,413 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:30,413 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:30,413 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All Shards are ready - data store testTransactionWithCreateTxFailureDueToNoLeader is ready 01:51:30,413 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader updated: matchIndex: -1, nextIndex: 0 01:51:30,413 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:30,423 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7da0615c: creating a new snapshot member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 01:51:30,424 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} 01:51:30,424 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:30,424 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:30,424 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-1444889622] 01:51:30,424 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, history=0, cookie=0} shard 1 01:51:30,424 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 01:51:30,424 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:30,424 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#544805759], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@1d0dfde2 01:51:30,424 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} scheduling timeout in 2 seconds 01:51:30,424 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:30,424 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, primaryVersion=13] 01:51:30,426 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, primaryVersion=13] 01:51:30,427 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6f35fa0e, attempting to connect 01:51:30,428 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} is not yet known 01:51:30,428 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, nanosAgo=10860, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:51:30,430 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:30,430 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:30,431 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:30,431 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@190ba14 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@31b7c3d2 01:51:30,431 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:51:30,431 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@31b7c3d2 01:51:30,431 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1 01:51:30,431 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} 01:51:30,431 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#544805759], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} 01:51:30,431 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:30,431 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#544805759], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1247/0x0000000801301400@495af206 01:51:30,431 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1993187482 nanoseconds 01:51:30,431 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2d6add67 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#544805759], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=844755077511} 01:51:30,431 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7da0615c finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:30,431 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} 01:51:30,431 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@224c324b 01:51:30,431 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4c8a6b30 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@525e7a65, splicing queue 01:51:30,431 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4c8a6b30 queue spliced 0 messages 01:51:30,431 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 612.6 μs 01:51:30,433 PM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: allocating new open snapshot member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1 01:51:30,433 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-617: slice: identifier: member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0}} 01:51:30,433 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-617: Message size: 239 bytes, total slices to send: 1 01:51:30,433 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-617: Message does not need to be sliced - sending original message 01:51:30,434 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#544805759], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0}} 01:51:30,434 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#544805759], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=844755077511, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0}} 01:51:30,434 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0} 01:51:30,434 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:51:30,435 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:30,435 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader)_slicer-id-620: Closing 01:51:30,435 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Aborting 0 pending queued transactions 01:51:30,435 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:51:30,438 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:30,438 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:30,440 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:30,440 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:30,440 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:30,440 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:30,455 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 01:51:30,555 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithCreateTxFailureDueToNoLeader 01:51:30,555 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7da0615c: creating a new transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 01:51:30,555 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 01:51:30,555 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:51:30,555 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7da0615c readied transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 01:51:30,555 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:30,555 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#544805759], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@1deecb19 01:51:30,555 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 01:51:30,555 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:30,555 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:30,555 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2d6add67 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#544805759], modifications=1, protocol=SIMPLE}, enqueuedTicks=844886329931} 01:51:30,555 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 01:51:30,555 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-618: slice: identifier: member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#544805759], modifications=1, protocol=SIMPLE}} 01:51:30,555 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : true 01:51:30,555 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 01:51:30,556 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-618: Message size: 401 bytes, total slices to send: 1 01:51:30,556 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-618: Message does not need to be sliced - sending original message 01:51:30,687 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 01:51:30,687 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 01:51:30,687 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2559, Joining) 01:51:30,687 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Received ElectionTimeout - switching to Candidate 01:51:30,687 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 01:51:30,687 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:30,687 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 5 01:51:30,687 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:30,687 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:51:30,687 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 01:51:30,687 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:51:30,687 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 01:51:30,687 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:51:30,687 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Follower to Candidate 01:51:30,687 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=null, leaderPayloadVersion=-1] 01:51:30,688 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:30,688 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Follower to Candidate 01:51:30,688 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:30,688 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2d6add67 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@292404ac, splicing queue 01:51:30,688 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1867558410 nanoseconds 01:51:30,688 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2d6add67 queue spliced 1 messages 01:51:30,688 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:30,688 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: refreshing backend for shard 1 01:51:30,688 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:30,688 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:30,688 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:30,688 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-1444889622] 01:51:30,688 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:30,688 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Candidate, leaderId=null, activeMember=true] 01:51:30,877 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:51:30,877 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=6, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:30,877 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 6 01:51:30,877 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:30,906 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received ShardNotInitializedTimeout message for shard cars 01:51:30,906 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Returning NoShardLeaderException for shard cars 01:51:30,906 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:51:30,907 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:30,907 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:30,907 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:31,96 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:51:31,96 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=7, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:31,96 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 7 01:51:31,96 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=7, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:31,297 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:51:31,297 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=8, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:31,297 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 8 01:51:31,297 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=8, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:31,456 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [1567168438696134565] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:31,457 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [1567168438696134565] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:31,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:51:31,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=9, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:31,517 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 9 01:51:31,517 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=9, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:31,727 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:51:31,727 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=10, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:31,727 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 10 01:51:31,727 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=10, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:31,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:51:31,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=11, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:31,886 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 11 01:51:31,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=11, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:31,927 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:31,927 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:31,927 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:31,927 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-1444889622] 01:51:31,927 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:31,927 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Candidate, leaderId=null, activeMember=true] 01:51:32,17 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:51:32,17 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=12, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:32,17 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 12 01:51:32,17 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=12, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:32,146 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received ShardNotInitializedTimeout message for shard cars 01:51:32,146 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Returning NoShardLeaderException for shard cars 01:51:32,146 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:51:32,147 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:32,147 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:32,147 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:32,156 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:51:32,156 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=13, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:32,156 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 13 01:51:32,156 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=13, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:32,377 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:51:32,377 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=14, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:32,377 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 14 01:51:32,377 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=14, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:32,436 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: running timer on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} 01:51:32,437 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} is empty 01:51:32,437 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: not scheduling timeout on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} 01:51:32,447 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: running timer on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:32,447 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} is empty 01:51:32,447 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: not scheduling timeout on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:32,536 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 01:51:32,536 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=15, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:32,537 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 15 01:51:32,537 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=15, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:32,577 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: running timer on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:32,577 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} timed out entry TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#544805759], modifications=1, protocol=SIMPLE}, enqueuedTicks=844886329931, sessionId=0, txSequence=1} 01:51:32,579 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} timed out 1 tasks 01:51:32,579 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: not scheduling timeout on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:32,581 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=FLUSHED} directCommit completed 01:51:32,581 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=FLUSHED}: initiating purge 01:51:32,581 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=DONE} allocated sequence 1 01:51:32,581 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#544805759]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@cf07445 01:51:32,581 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} has a successor, not scheduling timer 01:51:32,581 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1999952149 nanoseconds 01:51:32,581 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:51:32,581 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithCreateTxFailureDueToNoLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:51:32,581 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:51:32,581 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 01:51:32,582 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithCreateTxFailureDueToNoLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:51:32,582 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:51:32,582 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 01:51:32,582 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 01:51:32,582 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Aborting 0 pending queued transactions 01:51:32,582 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 01:51:32,583 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:32,583 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:33,595 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-6118546351265688535] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:33,598 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:33,598 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:33,599 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:33.601] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:33,608 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:33,608 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:33,614 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:33.615] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:33,632 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:51:33,638 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-8534405092310458429] 01:51:33,638 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:51:33,639 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:51:33,639 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:51:33,643 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:33,643 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:33,651 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:51:33,651 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:51:33,651 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:51:33,655 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:51:33,659 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [7449295008996783549] 01:51:33,659 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:51:33,662 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:33,662 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:51:33,665 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:33,665 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:33,676 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:33,688 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-259850911272177318] 01:51:33,688 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:51:33,690 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:33,690 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:51:33,695 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:33,695 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:33,701 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:33,703 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:33,704 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithMultipleShards 01:51:33,707 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-1157530447] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:33,708 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 01:51:33,715 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:51:33,716 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: no initial generation override, starting from 0 01:51:33,716 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:51:33,716 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@36f2bc94 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5a445251 01:51:33,716 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1f46d77d 01:51:33,716 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithMultipleShards 01:51:33,716 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:33,717 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:33,717 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:33,717 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:33,717 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:33,719 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:33,719 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, 0, 1710165093719, None)) 01:51:33,720 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165093718,0,0)) 01:51:33,720 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5a445251 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4e608387 01:51:33,720 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} started 01:51:33,720 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:33,722 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:33,722 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@15fc78f4 01:51:33,722 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:33,722 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:33,722 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:33,722 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:33,722 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#1507602534] 01:51:33,722 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:33,722 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:33,722 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:33,722 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards on shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-412145762] 01:51:33,722 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, persistent : true 01:51:33,723 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with journal batch size 1 01:51:33,723 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithMultipleShards 01:51:33,723 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-people-testChainedTransactionFailureWithMultipleShards, persistent : true 01:51:33,723 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-people-testChainedTransactionFailureWithMultipleShards with journal batch size 1 01:51:33,723 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#1507602534] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:33,723 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#1969840013 created and ready for shard:member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:33,723 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-412145762] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:33,724 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#211676398] 01:51:33,724 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#1969840013] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:33,724 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#2101480422 created and ready for shard:member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:33,724 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-1742725360] 01:51:33,724 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#211676398] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:33,726 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#2101480422] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:33,726 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-1742725360] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:33,727 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:33,727 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:33,728 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:51:33,730 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 01:51:33,732 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:33,733 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [people] 01:51:33,733 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is initialized 01:51:33,733 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:33,733 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:33,734 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [cars] 01:51:33,734 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is initialized 01:51:33,735 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:33,735 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:51:33,736 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 01:51:33,736 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 01:51:33,736 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from null to Follower 01:51:33,736 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-523216920] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:33,738 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:33,738 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:51:33,738 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 01:51:33,739 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 01:51:33,739 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from null to Follower 01:51:33,740 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithMultipleShards 01:51:33,740 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: no initial generation override, starting from 0 01:51:33,740 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:51:33,740 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@622ec151 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6e3e582e 01:51:33,740 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:33,740 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6abc03e4 01:51:33,741 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:33,741 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:33,741 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:33,741 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:33,742 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, 0, 1710165093742, None)) 01:51:33,742 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165093741,0,0)) 01:51:33,742 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6e3e582e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1028ce1f 01:51:33,742 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} started 01:51:33,743 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@60698717 01:51:33,743 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:33,743 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:33,743 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:33,743 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:33,743 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#938636521] 01:51:33,743 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:33,743 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, persistent : true 01:51:33,744 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with journal batch size 1 01:51:33,744 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#938636521] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:33,744 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-1522780267 created and ready for shard:member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:33,744 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#100478046] 01:51:33,744 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-people-testChainedTransactionFailureWithMultipleShards, persistent : true 01:51:33,744 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-1522780267] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:33,744 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#-2009175239] 01:51:33,744 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#1057295697 created and ready for shard:member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:33,745 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:33,745 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:33,745 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards on shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards#1397590034] 01:51:33,745 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#100478046] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:33,745 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards#1397590034] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:33,745 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#1057295697] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:33,745 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#-2009175239] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:33,746 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-people-testChainedTransactionFailureWithMultipleShards with journal batch size 1 01:51:33,747 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:33,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:33,747 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:33,747 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:33,747 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [people] 01:51:33,747 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards is initialized 01:51:33,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:33,747 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:33,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:51:33,747 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:51:33,747 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [cars] 01:51:33,747 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards is initialized 01:51:33,747 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 01:51:33,747 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:33,747 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:33,747 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 01:51:33,747 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 01:51:33,747 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 01:51:33,748 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards from null to Follower 01:51:33,748 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-2-shard-people-testChainedTransactionFailureWithMultipleShards from null to Follower 01:51:33,793 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-1157530447] 01:51:33,793 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: findLocalShard : cars 01:51:33,793 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#1507602534] 01:51:33,960 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:33,960 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:51:33,960 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testChainedTransactionFailureWithMultipleShards] 01:51:33,960 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:33,960 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 1 01:51:33,960 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:33,960 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Closing 01:51:33,960 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:33,960 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:51:33,960 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from Follower to Candidate 01:51:33,961 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from Follower to Candidate 01:51:33,999 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:33,999 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:51:33,999 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testChainedTransactionFailureWithMultipleShards] 01:51:33,999 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:33,999 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 1 01:51:33,999 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:33,999 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Closing 01:51:33,999 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:33,999 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 01:51:33,999 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from Follower to Candidate 01:51:34,0 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from Follower to Candidate 01:51:34,199 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:51:34,199 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:34,199 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 2 01:51:34,229 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:51:34,229 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:34,230 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 2 01:51:34,281 PM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:34,281 PM [cluster-test-shard-dispatcher-30] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:34,430 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:51:34,430 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:34,430 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 3 01:51:34,540 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:51:34,540 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:34,540 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 3 01:51:34,651 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:34,651 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:34,651 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:34,651 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:51:34,651 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:34,651 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:51:34,651 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:34,652 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:34,652 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:34,652 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:34,652 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:34,652 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:34,652 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:34,652 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:34,657 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:34,657 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:34,657 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:34,657 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:34,657 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:34,657 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:34,657 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:34,657 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:34,657 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:34,657 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:34,703 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:51:34,739 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:51:34,739 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:34,740 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 4 01:51:34,740 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:34,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:34,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Cluster state: [] 01:51:34,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:34,741 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:34,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=null 01:51:34,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:34,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:34,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:34,742 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:34,742 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:34,743 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testChainedTransactionFailureWithMultipleShards] 01:51:34,743 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:34,743 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:34,743 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:51:34,743 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: became leader with frontend state for [] 01:51:34,743 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 01:51:34,743 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:34,743 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:34,743 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from Candidate to Leader 01:51:34,743 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from Candidate to Leader 01:51:34,743 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Closing 01:51:34,744 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testChainedTransactionFailureWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:34,744 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-people-testChainedTransactionFailureWithMultipleShards status sync done false 01:51:34,744 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 01:51:34,744 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:51:34,744 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:34,744 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:34,744 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:34,810 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 01:51:34,810 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:34,810 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 4 01:51:34,810 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:34,812 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:34,812 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Cluster state: [] 01:51:34,812 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:34,812 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:34,812 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=null 01:51:34,812 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:34,812 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:34,812 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:34,813 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:34,813 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:34,813 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testChainedTransactionFailureWithMultipleShards] 01:51:34,813 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:34,813 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:34,813 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:51:34,813 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: became leader with frontend state for [] 01:51:34,813 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from Candidate to Leader 01:51:34,813 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 01:51:34,813 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:34,813 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:34,813 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from Candidate to Leader 01:51:34,813 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All Shards are ready - data store testChainedTransactionFailureWithMultipleShards is ready 01:51:34,814 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Closing 01:51:34,814 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testChainedTransactionFailureWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:34,814 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards status sync done false 01:51:34,814 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 01:51:34,814 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:34,814 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:34,814 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All Shards are ready - data store testChainedTransactionFailureWithMultipleShards is ready 01:51:34,814 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:51:34,815 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:34,860 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-people-testChainedTransactionFailureWithMultipleShards status sync done true 01:51:34,898 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-1157530447] 01:51:34,898 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: findLocalShard : people 01:51:34,898 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-412145762] 01:51:34,899 PM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@7326d276 01:51:34,899 PM [main] [DEBUG] AbstractDOMBroker - Transaction chain 0 created with listener Mock for DOMTransactionChainListener, hashCode: 824742124, backing store chains {CONFIGURATION=org.opendaylight.controller.cluster.databroker.ClientBackedTransactionChain@5a30b0bf} 01:51:34,899 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@7326d276 changed state from IDLE to TX_OPEN 01:51:34,899 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} 01:51:34,899 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard people 01:51:34,899 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard people 01:51:34,899 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-523216920] 01:51:34,900 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} shard 1 01:51:34,900 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:51:34,900 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 01:51:34,900 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 01:51:34,900 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:51:34,900 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} 01:51:34,900 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard cars 01:51:34,900 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard cars 01:51:34,900 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-523216920] 01:51:34,900 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 01:51:34,900 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} shard 2 01:51:34,900 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:34,900 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} scheduling timeout in 30 seconds 01:51:34,900 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 01:51:34,900 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 for transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 01:51:34,900 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} allocated sequence 0 01:51:34,901 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@10fbb3d 01:51:34,901 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 01:51:34,901 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@60f5e391 01:51:34,901 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@7326d276 changed state from TX_OPEN to IDLE 01:51:34,901 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@7326d276 readied transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 01:51:34,901 PM [main] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 is submitted for execution. 01:51:34,901 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 01:51:34,901 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@6b62dbb1 01:51:34,901 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 01:51:34,901 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=FLUSHED} allocated sequence 1 01:51:34,901 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@39cf1854, attempting to connect 01:51:34,901 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@187272b0 01:51:34,901 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@394b5a68, attempting to connect 01:51:34,903 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} is not yet known 01:51:34,903 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, nanosAgo=11910, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:51:34,903 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} is not yet known 01:51:34,904 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, nanosAgo=12770, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:51:34,913 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:34,913 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:34,913 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-412145762], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent} 01:51:34,913 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-412145762], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} 01:51:34,913 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@581b78b0 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@232df8 01:51:34,913 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:51:34,913 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-412145762], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:51:34,913 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@21875be4 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574]}, enqueuedTicks=849230639757} 01:51:34,913 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@232df8 01:51:34,913 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:34,913 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} 01:51:34,913 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=1, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} 01:51:34,913 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 01:51:34,913 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2238/0x00000008016aba80@3895b6ba 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@21875be4 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=1, protocol=READY}, enqueuedTicks=849231684372} 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-624: slice: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=1, protocol=READY}} 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-624: Message size: 406 bytes, total slices to send: 1 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-624: Message does not need to be sliced - sending original message 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=0, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 1 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2239/0x00000008016abcb8@581b2820 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@21875be4 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=849231949609} 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-624: slice: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=0, protocol=THREE_PHASE}} 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-624: Message size: 298 bytes, total slices to send: 1 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-624: Message does not need to be sliced - sending original message 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@7326d276 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-412145762], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@7f233e26 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@75109bb8 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@25aabd25, splicing queue 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@75109bb8 queue spliced 0 messages 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-412145762], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} in 902.9 μs 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#1507602534], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent} 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#1507602534], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6b5c0806 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@16286f57 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} previous state FLUSHED 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#1507602534], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6794fb71 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574]}, enqueuedTicks=849231412586} 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@16286f57 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 for transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=1, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} allocated sequence 0 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2238/0x00000008016aba80@2458976 01:51:34,914 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6794fb71 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=1, protocol=READY}, enqueuedTicks=849231759634} 01:51:34,915 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-624: slice: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, message: Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=1, protocol=READY}} 01:51:34,915 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-624: Message size: 491 bytes, total slices to send: 1 01:51:34,915 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-624: Message does not need to be sliced - sending original message 01:51:34,915 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=0, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} 01:51:34,915 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 1 01:51:34,915 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2239/0x00000008016abcb8@7e88efdc 01:51:34,915 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6794fb71 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=849232055192} 01:51:34,915 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-624: slice: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, message: Envelope{sessionId=1, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=0, protocol=THREE_PHASE}} 01:51:34,915 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-624: Message size: 299 bytes, total slices to send: 1 01:51:34,915 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-624: Message does not need to be sliced - sending original message 01:51:34,915 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@7326d276 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#1507602534], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} 01:51:34,915 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 01:51:34,915 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@749b2305 01:51:34,915 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6b74daa2 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@322fb53d, splicing queue 01:51:34,915 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6b74daa2 queue spliced 0 messages 01:51:34,915 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#1507602534], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} in 697.8 μs 01:51:34,917 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 01:51:34,917 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 01:51:34,917 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 01:51:34,917 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 01:51:34,918 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:51:34,918 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:51:34,918 PM [cluster-test-shard-dispatcher-32] [DEBUG] LeaderFrontendState - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 01:51:34,918 PM [cluster-test-shard-dispatcher-25] [DEBUG] LeaderFrontendState - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 01:51:34,918 PM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: allocating new open transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:34,918 PM [cluster-test-shard-dispatcher-25] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: allocating new open transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 01:51:34,918 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 01:51:34,918 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 01:51:34,918 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@41ff2ad1 01:51:34,918 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@74cdf900 01:51:34,918 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: readying transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:34,918 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 ready 01:51:34,918 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE}}}]} 01:51:34,918 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: transitioned member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 to ready 01:51:34,918 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendTransaction - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#2084325574], modifications=1, protocol=READY} failed to process org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:51:34,918 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 is already in state READY (READY) 01:51:34,918 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 initiating canCommit 01:51:34,918 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Validating transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:34,918 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 validated 01:51:34,918 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:34,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 completed canCommit 01:51:34,919 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: request Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#2084325574], modifications=1, protocol=READY}} failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#2084325574], modifications=1, protocol=READY} failed to process at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:147) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ... 33 more 01:51:34,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 01:51:34,920 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: request Envelope{sessionId=1, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#2084325574], modifications=0, protocol=THREE_PHASE}} failed org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:51:34,920 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 01:51:34,928 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0}} 01:51:34,928 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=1, protocol=READY}, enqueuedTicks=849231684372, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0}} 01:51:34,928 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0} 01:51:34,928 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1}} 01:51:34,928 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=849231949609, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1}} 01:51:34,928 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} canCommit completed 01:51:34,929 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=1, protocol=READY} with Envelope{sessionId=1, txSequence=1, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#2084325574], modifications=1, protocol=READY} failed to process}} 01:51:34,929 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=1, protocol=READY}, enqueuedTicks=849231759634, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#2084325574], modifications=1, protocol=READY} failed to process}} 01:51:34,929 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=1, protocol=READY} completed with TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#2084325574], modifications=1, protocol=READY} failed to process} 01:51:34,929 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteProxyTransaction - Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#2084325574], modifications=1, protocol=READY} failed to process at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:147) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ... 33 more 01:51:34,929 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=2, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 01:51:34,929 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2084325574], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=849232055192, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 01:51:34,929 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during phase CAN_COMMIT, starting Abort org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:51:34,929 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DOMBrokerTransactionChain - Transaction chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@2feaf4e5 failed. TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0]]} at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1127) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) at org.opendaylight.controller.cluster.databroker.actors.dds.VotingFuture.resolveResult(VotingFuture.java:91) at org.opendaylight.controller.cluster.databroker.actors.dds.VotingFuture.voteNo(VotingFuture.java:67) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction.lambda$canCommit$5(AbstractProxyTransaction.java:549) at org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction.lambda$handleReplayedModifyTransactionRequest$17(RemoteProxyTransaction.java:516) at org.opendaylight.controller.cluster.access.client.ConnectionEntry.complete(ConnectionEntry.java:47) at org.opendaylight.controller.cluster.access.client.AbstractClientConnection.receiveResponse(AbstractClientConnection.java:494) at org.opendaylight.controller.cluster.access.client.AbstractReceivingClientConnection.receiveResponse(AbstractReceivingClientConnection.java:68) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onResponse(ClientActorBehavior.java:184) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onRequestFailure(ClientActorBehavior.java:196) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.internalOnRequestFailure(ClientActorBehavior.java:248) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:155) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:48) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.onReceiveCommand(AbstractClientActor.java:74) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) ... 17 more 01:51:34,929 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:34,929 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}]]] 01:51:34,930 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:51:34,930 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} sending request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2084325574]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1245/0x0000000801301800@39de6313 01:51:34,930 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} 01:51:34,930 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 2 01:51:34,930 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} sending request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2084325574]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1245/0x0000000801301800@39de6313 01:51:34,930 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@21875be4 transmitting entry ConnectionEntry{request=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2084325574]}, enqueuedTicks=849260775791} 01:51:34,930 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 2 01:51:34,930 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} sending request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2084325574]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1245/0x0000000801301800@6c6b0543 01:51:34,930 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} 01:51:34,930 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 2 01:51:34,930 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} sending request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2084325574]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1245/0x0000000801301800@6c6b0543 01:51:34,930 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6794fb71 transmitting entry ConnectionEntry{request=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2084325574]}, enqueuedTicks=849260902144} 01:51:34,930 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} canCommit completed 01:51:34,934 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 aborting 01:51:34,934 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: request Envelope{sessionId=1, txSequence=3, message=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#2084325574]}} failed org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:51:34,934 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}]]] 01:51:34,934 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 01:51:34,934 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:34,934 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77} 01:51:34,934 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 01:51:34,934 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 1 01:51:34,934 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:34,935 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: aborting head of queue member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 in state member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:34,935 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1: abort - successfulFromPrevious: [] 01:51:34,935 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 aborted 01:51:34,936 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 01:51:34,936 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards status sync done true 01:51:34,937 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2084325574]} with Envelope{sessionId=1, txSequence=3, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 01:51:34,938 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2084325574]}, enqueuedTicks=849260902144, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 01:51:34,938 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} abort completed 01:51:34,938 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR}: initiating purge 01:51:34,938 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 3 01:51:34,938 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2084325574]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@79f6e077 01:51:34,938 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} 01:51:34,938 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED}: initiating purge 01:51:34,938 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} allocated sequence 3 01:51:34,938 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:51:34,938 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:51:34,938 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 0 01:51:34,938 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 01:51:34,938 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77} 01:51:34,938 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2241/0x00000008016af5d0@6b15523e 01:51:34,938 PM [cluster-test-shard-dispatcher-25] [DEBUG] LeaderFrontendState - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 01:51:34,938 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2084325574]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@337ac2a1 01:51:34,938 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@6c57ca3c}, purged=MutableUnsignedLongSet{size=0}} 01:51:34,938 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6794fb71 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2084325574]}, enqueuedTicks=849268615704} 01:51:34,938 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 01:51:34,938 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2084325574]} with Envelope{sessionId=0, txSequence=3, message=TransactionAbortSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2}} 01:51:34,938 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:34,938 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2084325574]}, enqueuedTicks=849260775791, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionAbortSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2}} 01:51:34,938 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:51:34,938 PM [cluster-test-akka.actor.default-dispatcher-31] [ERROR] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during Abort. org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:51:34,939 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} abort completed 01:51:34,939 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: initiating purge 01:51:34,939 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 3 01:51:34,939 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2084325574]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@66e940d8 01:51:34,939 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} 01:51:34,939 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED}: initiating purge 01:51:34,939 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 3 01:51:34,939 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2084325574]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@3e3e96e 01:51:34,939 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@21875be4 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2084325574]}, enqueuedTicks=849269598428} 01:51:34,940 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: purging transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 01:51:34,940 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: purging transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:34,940 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78}] 01:51:34,940 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78}] 01:51:34,940 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:34,940 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}] 01:51:34,940 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}] 01:51:34,940 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:34,940 PM [main] [DEBUG] DOMBrokerTransactionChain - Chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@2feaf4e5 is no longer running 01:51:34,940 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:51:34,940 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574]} with Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} 01:51:34,940 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574]}, enqueuedTicks=849231412586, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} 01:51:34,940 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0} 01:51:34,940 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78}] 01:51:34,940 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}] 01:51:34,940 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:34,941 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:34,941 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onShutDown 01:51:34,941 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Initiating leader transfer 01:51:34,941 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards - waiting on 2 shards 01:51:34,941 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onShutDown 01:51:34,941 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Initiating leader transfer 01:51:34,941 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1452af2e 01:51:34,941 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@78556c07 01:51:34,941 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 01:51:34,941 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1452af2e 01:51:34,941 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Attempting to transfer leadership 01:51:34,941 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 01:51:34,941 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@78556c07 01:51:34,941 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Attempting to transfer leadership 01:51:34,941 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:51:34,941 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:51:34,941 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:34,941 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:34,941 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:51:34,941 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}]]] 01:51:34,941 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78}]]] 01:51:34,941 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:34,941 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:34,941 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:51:34,941 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:34,941 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:34,941 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 0 01:51:34,942 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:51:34,942 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 0 01:51:34,942 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 01:51:34,942 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77} 01:51:34,942 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@28cc555f}, purged=MutableUnsignedLongSet{size=0}} 01:51:34,942 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 01:51:34,942 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:51:34,942 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards] 01:51:34,942 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards] 01:51:34,942 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}]]] 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 2 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77} 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 01:51:34,943 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78} 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}] 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 2 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:34,943 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6794fb71 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@78532a8a, splicing queue 01:51:34,943 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#1507602534], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78}]]] 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:51:34,943 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6794fb71 queue spliced 1 messages 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78} 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78}] 01:51:34,943 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#1507602534], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#1507602534], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 2 01:51:34,943 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:34,943 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, matchIndex: 0, lastIndex: 1, isVoting: true 01:51:34,943 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: refreshing backend for shard 2 01:51:34,943 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#1507602534], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent} 01:51:34,943 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard cars 01:51:34,943 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard cars 01:51:34,943 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-523216920] 01:51:34,943 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:34,943 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@21875be4 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@778e6b4a, splicing queue 01:51:34,943 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-412145762], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:51:34,943 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@21875be4 queue spliced 2 messages 01:51:34,943 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78}] 01:51:34,943 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-412145762], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-412145762], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} 01:51:34,943 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: refreshing backend for shard 1 01:51:34,943 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#-412145762], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent} 01:51:34,943 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard people 01:51:34,943 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard people 01:51:34,943 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-523216920] 01:51:34,943 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 01:51:34,943 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}] 01:51:34,944 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:51:34,944 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 0 01:51:34,944 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 01:51:34,944 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:51:34,944 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 1 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77} 01:51:34,944 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 01:51:34,944 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78} 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2241/0x00000008016af5d0@40c8e025 01:51:34,944 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: replication of member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@37584d3 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] LeaderFrontendState - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 01:51:34,944 PM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: finished purging transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@287ac7c4}, purged=MutableUnsignedLongSet{size=0}} 01:51:34,944 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Purged transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 01:51:34,944 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 1 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78} 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: replication of member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@3ec90f0b 01:51:34,944 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: finished purging transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:34,944 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:51:34,944 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Purged transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:51:34,944 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:34,944 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Leader transfer complete 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testChainedTransactionFailureWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:51:34,944 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Leader transfer complete 01:51:34,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:51:34,945 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 1 01:51:34,945 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:51:34,945 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 1 01:51:34,945 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 01:51:34,945 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78} 01:51:34,945 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Purged transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 01:51:34,945 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:51:34,945 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 1 01:51:34,945 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 01:51:34,945 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:51:34,945 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 0 01:51:34,945 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 01:51:34,945 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77} 01:51:34,946 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5507c98f}, purged=MutableUnsignedLongSet{size=0}} 01:51:34,946 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 01:51:34,946 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 1 01:51:34,946 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 01:51:34,946 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78} 01:51:34,946 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} not found in inflight queue, checking pending queue 01:51:34,946 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Purged transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 01:51:34,946 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} 01:51:34,946 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:51:34,946 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2084325574]}, enqueuedTicks=849230639757, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} 01:51:34,946 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0} 01:51:34,946 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3}} not found in inflight queue, checking pending queue 01:51:34,946 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:51:34,946 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2084325574]} with Envelope{sessionId=1, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3}} 01:51:34,946 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2084325574]}, enqueuedTicks=849268615704, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3}} 01:51:34,946 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE}: purge completed 01:51:34,946 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@16286f57 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} 01:51:34,946 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR}: purge completed 01:51:34,946 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6b5c0806 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 01:51:34,946 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3}} not found in inflight queue, checking pending queue 01:51:34,946 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2084325574]} with Envelope{sessionId=0, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3}} 01:51:34,946 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2084325574]}, enqueuedTicks=849269598428, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3}} 01:51:34,946 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 01:51:34,946 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@232df8 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} 01:51:34,946 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: purge completed 01:51:34,946 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@581b78b0 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 01:51:35,360 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All shards shutdown - sending PoisonPill to self 01:51:35,360 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#1507602534]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:51:35,360 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:35,360 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader)_slicer-id-627: Closing 01:51:35,360 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 01:51:35,360 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:35,360 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader)_slicer-id-628: Closing 01:51:35,360 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 01:51:35,361 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 01:51:35,361 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 01:51:35,361 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 01:51:35,361 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 01:51:35,361 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 01:51:35,362 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:35,362 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:35,367 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:35,368 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:35,368 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:35,368 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:35,370 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:51:35,382 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 01:51:35,382 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 01:51:35,382 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Closing 01:51:35,382 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 01:51:35,382 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 01:51:35,382 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Closing 01:51:35,382 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 01:51:35,383 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 01:51:35,383 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:35,383 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:36,382 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-8534405092310458429] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:36,403 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [7449295008996783549] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:36,403 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:36,403 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:36,406 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:36.408] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:36,416 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:36,416 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:36,416 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:36.417] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:36,429 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:36,434 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [6123523105792681763] 01:51:36,434 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:51:36,436 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:51:36,436 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:51:36,437 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:36,437 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:36,445 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@388db054 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} 01:51:36,445 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} allocated sequence 1 01:51:36,445 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@3eefca 01:51:36,445 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:36,445 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7c8ff06b transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=0, protocol=ABORT}, enqueuedTicks=850776166726} 01:51:36,445 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-608: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=0, protocol=ABORT}} 01:51:36,445 PM [Cleaner-1] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-608: Message size: 286 bytes, total slices to send: 1 01:51:36,445 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-608: Message does not need to be sliced - sending original message 01:51:36,445 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0} 01:51:36,445 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@608d22bc aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 01:51:36,445 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:51:36,445 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-43977299], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@7b956a43 01:51:36,445 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} 01:51:36,445 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@c253bdd aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 01:51:36,445 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-43977299]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@3a997a33 01:51:36,446 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:51:36,446 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@388db054 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} 01:51:36,446 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} allocated sequence 1 01:51:36,446 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@332a0d64 01:51:36,446 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7c8ff06b transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=0, protocol=ABORT}, enqueuedTicks=850776644508} 01:51:36,446 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-608: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, message: Envelope{sessionId=0, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=0, protocol=ABORT}} 01:51:36,446 PM [Cleaner-1] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-608: Message size: 286 bytes, total slices to send: 1 01:51:36,446 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-608: Message does not need to be sliced - sending original message 01:51:36,446 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0} 01:51:36,446 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@388db054 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} 01:51:36,446 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} allocated sequence 1 01:51:36,446 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@4ddcd821 01:51:36,446 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7c8ff06b transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=0, protocol=ABORT}, enqueuedTicks=850776758221} 01:51:36,446 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-608: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, message: Envelope{sessionId=0, txSequence=9, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1868627509], modifications=0, protocol=ABORT}} 01:51:36,446 PM [Cleaner-1] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-608: Message size: 286 bytes, total slices to send: 1 01:51:36,446 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-608: Message does not need to be sliced - sending original message 01:51:36,446 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0} 01:51:36,446 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@c253bdd aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} 01:51:36,446 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-43977299]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@622a9546 01:51:36,446 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#422429231], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:51:36,446 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@50e50a96 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} 01:51:36,446 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:51:36,446 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=1, replyTo=Actor[akka://reinstated-member2/user/$a#-756160416], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@7b529e28 01:51:36,446 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} 01:51:36,446 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@780b5c28 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} 01:51:36,446 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=0, replyTo=Actor[akka://reinstated-member2/user/$a#-756160416]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@7fcbabfb 01:51:36,446 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#210029802], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:51:36,446 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@190ba14 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} 01:51:36,446 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:51:36,446 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#544805759], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@5ab78e40 01:51:36,446 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} has a successor, not scheduling timer 01:51:36,446 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SEALED} allocated sequence 1 01:51:36,446 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#544805759], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1240/0x0000000801303148@6c01a0f5 01:51:36,446 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1481151261], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} has a successor, not scheduling timer 01:51:36,446 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0} 01:51:36,454 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:51:36,454 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:51:36,454 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:51:36,456 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:51:36,464 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [6628146650783934260] 01:51:36,464 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:51:36,466 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:36,466 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:51:36,472 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:36,472 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:36,479 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:51:36,487 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [1303849705611730207] 01:51:36,487 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:51:36,493 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:36,493 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:51:36,497 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:36,499 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:36,499 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:36,499 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:36,501 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 01:51:36,505 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 01:51:36,507 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-553692733] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:36,509 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 01:51:36,509 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:51:36,509 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@241b2616 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@483574ce 01:51:36,509 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@27360eb6 01:51:36,510 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1710165096510, None)) 01:51:36,510 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 01:51:36,510 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:36,510 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-1-shard-people-testLeadershipTransferOnShutdown 01:51:36,510 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:36,510 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:36,510 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:51:36,510 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165096509,0,0)) 01:51:36,511 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@483574ce to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7f9b952b 01:51:36,511 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 01:51:36,512 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:36,513 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:36,515 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 01:51:36,515 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1c2316ab 01:51:36,515 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:36,515 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:36,516 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:36,516 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:36,516 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-1-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851] 01:51:36,516 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testLeadershipTransferOnShutdown 01:51:36,516 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:51:36,516 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:51:36,516 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-1-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-2045441789] 01:51:36,516 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-1-shard-cars-testLeadershipTransferOnShutdown, persistent : true 01:51:36,516 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-people-testLeadershipTransferOnShutdown, persistent : true 01:51:36,518 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 01:51:36,518 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 01:51:36,518 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1350683196] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:36,519 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:36,519 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-2045441789] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:36,519 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-notifier#497268843 created and ready for shard:member-1-shard-people-testLeadershipTransferOnShutdown 01:51:36,519 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#756294161 created and ready for shard:member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:36,520 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#42162963] 01:51:36,520 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-notifier#497268843] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:36,520 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-2068149328] 01:51:36,522 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-1-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 01:51:36,522 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:36,522 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:36,522 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:51:36,522 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:51:36,522 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 01:51:36,522 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 01:51:36,522 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-people-testLeadershipTransferOnShutdown is initialized 01:51:36,524 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:51:36,524 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:36,524 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:36,524 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:51:36,524 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:51:36,525 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 01:51:36,526 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 01:51:36,526 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:51:36,526 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5f73e4fb to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@74be3e96 01:51:36,526 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7081b4d9 01:51:36,526 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1710165096526, None)) 01:51:36,526 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165096525,0,0)) 01:51:36,527 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@74be3e96 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5b2150e0 01:51:36,527 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 01:51:36,527 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 01:51:36,527 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-2-shard-cars-testLeadershipTransferOnShutdown 01:51:36,527 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-2-shard-people-testLeadershipTransferOnShutdown 01:51:36,527 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from null to Follower 01:51:36,527 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 01:51:36,527 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-testLeadershipTransferOnShutdown is initialized 01:51:36,527 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@500bbde6 01:51:36,527 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:36,527 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testLeadershipTransferOnShutdown 01:51:36,527 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:51:36,528 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from null to Follower 01:51:36,530 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#756294161] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:36,530 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#42162963] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:36,530 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-2068149328] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:36,530 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testLeadershipTransferOnShutdown 01:51:36,531 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testLeadershipTransferOnShutdown, persistent : true 01:51:36,531 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:36,531 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-notifier#-45873132 created and ready for shard:member-2-shard-cars-testLeadershipTransferOnShutdown 01:51:36,531 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-notifier#-45873132] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:36,531 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 01:51:36,531 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1454008181] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:36,531 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-people-testLeadershipTransferOnShutdown, persistent : true 01:51:36,531 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-737233212] 01:51:36,531 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#113232257] 01:51:36,532 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-notifier#1302776446] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:36,532 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:36,532 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:36,532 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:51:36,532 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:51:36,532 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 01:51:36,532 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is initialized 01:51:36,532 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-notifier#1302776446 created and ready for shard:member-2-shard-people-testLeadershipTransferOnShutdown 01:51:36,533 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 01:51:36,533 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:51:36,533 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 01:51:36,533 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from null to Follower 01:51:36,536 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:36,536 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:36,536 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 01:51:36,536 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized 01:51:36,536 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:51:36,536 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:51:36,536 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-737233212] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:36,536 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#113232257] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:36,536 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 01:51:36,536 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:51:36,538 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from null to Follower 01:51:36,544 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:51:36,544 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:51:36,546 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:36,554 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:36,554 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:36,554 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:36,554 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:51:36,554 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:51:36,554 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:51:36,554 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:36,554 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:51:36,585 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-553692733] 01:51:36,585 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 01:51:36,586 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851] 01:51:36,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:36,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 01:51:36,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-3-shard-cars-testLeadershipTransferOnShutdown] 01:51:36,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:36,747 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 01:51:36,747 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:36,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 01:51:36,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:51:36,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:51:36,747 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:51:36,747 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 01:51:36,797 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:36,797 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 01:51:36,797 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 01:51:36,797 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 01:51:36,797 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 01:51:36,797 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:36,797 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 01:51:36,797 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:51:36,797 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:51:36,797 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:51:36,798 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 01:51:37,46 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:51:37,46 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=2, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:37,47 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 2 01:51:37,77 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:51:37,77 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=2, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 01:51:37,77 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 2 01:51:37,306 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:51:37,306 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:37,307 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 3 01:51:37,327 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:51:37,327 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 01:51:37,327 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 3 01:51:37,457 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:51:37,457 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:37,457 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:51:37,457 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,457 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:37,457 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:51:37,457 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:51:37,457 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:37,457 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:51:37,457 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:37,457 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:51:37,457 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:51:37,457 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-3-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,458 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:51:37,458 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-3-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:51:37,458 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-2-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:51:37,510 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:37,510 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,510 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:51:37,510 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:51:37,510 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:51:37,510 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:37,510 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:51:37,510 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:51:37,510 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:51:37,510 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:51:37,510 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-3-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,510 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-3-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:51:37,518 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:51:37,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 01:51:37,565 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 01:51:37,565 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 01:51:37,565 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 01:51:37,565 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 01:51:37,565 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=1, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,565 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=1, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:37,565 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:37,565 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 01:51:37,565 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:51:37,565 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:51:37,565 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:51:37,565 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 01:51:37,566 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:37,571 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVote [term=1, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1], my term is 3 01:51:37,571 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): In requestVote: RequestVote [term=1, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: member-1-shard-cars-testLeadershipTransferOnShutdown, lastIndex: -1, lastTerm: -1 01:51:37,571 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): requestVote returning: RequestVoteReply [term=3, voteGranted=false] 01:51:37,572 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=false], my term is 1 01:51:37,572 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Term 3 in "RequestVoteReply [term=3, voteGranted=false]" message is greater than Candidate's term 1 - switching to Follower 01:51:37,572 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=null 01:51:37,572 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Follower, election term: 3 01:51:37,572 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:51:37,572 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:51:37,572 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Follower 01:51:37,572 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Candidate to Follower 01:51:37,575 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 01:51:37,575 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 01:51:37,575 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 01:51:37,575 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 01:51:37,575 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 01:51:37,575 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=1, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-people-testLeadershipTransferOnShutdown 01:51:37,575 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=1, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-testLeadershipTransferOnShutdown 01:51:37,575 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:37,575 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 01:51:37,575 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:51:37,575 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:51:37,575 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:51:37,576 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 01:51:37,576 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVote [term=1, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1], my term is 3 01:51:37,576 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): In requestVote: RequestVote [term=1, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: member-1-shard-people-testLeadershipTransferOnShutdown, lastIndex: -1, lastTerm: -1 01:51:37,576 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): requestVote returning: RequestVoteReply [term=3, voteGranted=false] 01:51:37,577 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1454008181] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:37,578 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:51:37,578 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 01:51:37,578 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 4 01:51:37,578 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testLeadershipTransferOnShutdown 01:51:37,578 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-testLeadershipTransferOnShutdown 01:51:37,579 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=false], my term is 1 01:51:37,579 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Term 3 in "RequestVoteReply [term=3, voteGranted=false]" message is greater than Candidate's term 1 - switching to Follower 01:51:37,579 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=null 01:51:37,579 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Follower, election term: 3 01:51:37,579 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:51:37,580 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:51:37,580 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Follower 01:51:37,580 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Candidate to Follower 01:51:37,580 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-2045441789] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:37,581 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:37,581 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 01:51:37,581 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:37,581 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 3 - updating term 01:51:37,581 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 01:51:37,581 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:37,581 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 01:51:37,581 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:37,585 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:37,585 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:37,585 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 01:51:37,585 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:37,586 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:37,586 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:37,586 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 01:51:37,586 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: became leader with frontend state for [] 01:51:37,586 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 01:51:37,586 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:51:37,586 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:37,586 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:37,586 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Candidate to Leader 01:51:37,587 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown: Closing 01:51:37,587 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:51:37,587 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-people-testLeadershipTransferOnShutdown status sync done false 01:51:37,587 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:51:37,587 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-2045441789] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:37,587 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:37,587 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:37,588 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 01:51:37,588 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:37,627 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 01:51:37,627 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:37,627 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 4 01:51:37,627 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testLeadershipTransferOnShutdown 01:51:37,627 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,628 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:37,628 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 01:51:37,628 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:37,628 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 3 - updating term 01:51:37,628 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 01:51:37,628 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:37,628 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:37,628 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:37,628 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:37,629 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:37,629 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:37,629 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-3-shard-cars-testLeadershipTransferOnShutdown] 01:51:37,629 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:37,629 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:37,629 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:37,629 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 01:51:37,629 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [] 01:51:37,630 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 01:51:37,630 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:51:37,630 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:37,630 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:37,630 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 01:51:37,630 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:51:37,630 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 01:51:37,630 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:37,630 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:51:37,630 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:37,630 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:37,630 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:51:37,630 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:37,630 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done false 01:51:37,631 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 01:51:37,631 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:37,692 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-553692733] 01:51:37,692 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : people 01:51:37,692 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-2045441789] 01:51:37,693 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:37,695 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:37,697 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 01:51:37,700 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1875823032] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:37,701 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 01:51:37,701 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-2045441789] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:37,701 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-people-testLeadershipTransferOnShutdown status sync done true 01:51:37,703 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 01:51:37,703 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 01:51:37,703 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:51:37,703 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,703 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3a5ce93b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7b088293 01:51:37,703 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5cdfadf4 01:51:37,703 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-3-shard-people-testLeadershipTransferOnShutdown 01:51:37,703 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:37,703 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,703 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:51:37,703 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:37,703 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:37,703 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:51:37,703 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:37,703 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:51:37,703 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:51:37,703 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1710165097703, None)) 01:51:37,703 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165097702,0,0)) 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2fe186dd 01:51:37,704 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7b088293 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@b7eeb2e 01:51:37,704 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,704 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2ac9d9eb: creating a new transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-2100244782] 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-2-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-2100244782] 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-3-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-2100244782] 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-people-testLeadershipTransferOnShutdown 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#1720974242] 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-2-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#1720974242] 01:51:37,704 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:51:37,704 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 01:51:37,704 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:37,704 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-3-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#1720974242] 01:51:37,704 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1350683196] 01:51:37,704 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} shard 1 01:51:37,705 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:51:37,705 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:37,705 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-3-shard-cars-testLeadershipTransferOnShutdown, persistent : true 01:51:37,705 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-2100244782] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:37,705 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} 01:51:37,705 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:51:37,705 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:51:37,705 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1350683196] 01:51:37,705 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} shard 2 01:51:37,705 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:51:37,705 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:51:37,705 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2ac9d9eb readied transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:51:37,705 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2ac9d9eb: creating a new transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:51:37,705 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-notifier#1064465583 created and ready for shard:member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,705 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Shard created : member-3-shard-people-testLeadershipTransferOnShutdown, persistent : true 01:51:37,705 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:51:37,705 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:51:37,705 PM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:51:37,705 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2ac9d9eb readied transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 01:51:37,705 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-notifier#1064465583] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:37,705 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#-43914166] 01:51:37,705 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Starting recovery for member-3-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 01:51:37,705 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:37,705 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-3-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 01:51:37,705 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 01:51:37,705 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:51:37,706 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 01:51:37,706 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown, primaryVersion=13] 01:51:37,706 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown, primaryVersion=13] 01:51:37,706 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@1be8321c, attempting to connect 01:51:37,706 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@5645d08e, attempting to connect 01:51:37,707 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-notifier#312107545 created and ready for shard:member-3-shard-people-testLeadershipTransferOnShutdown 01:51:37,707 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testLeadershipTransferOnShutdown 01:51:37,707 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-553692733] 01:51:37,707 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#1191931141] 01:51:37,708 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} is not yet known 01:51:37,708 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=14031, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:51:37,708 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:37,708 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} is not yet known 01:51:37,708 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-people-testLeadershipTransferOnShutdown 01:51:37,708 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=4480, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:51:37,708 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 01:51:37,708 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 01:51:37,708 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 01:51:37,708 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851] 01:51:37,709 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], modifications=2, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@708c479a 01:51:37,709 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:51:37,709 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@caa468b 01:51:37,709 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@38ac4f16 skipping transactions [1] 01:51:37,709 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} scheduling timeout in 30 seconds 01:51:37,709 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:37,709 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:37,709 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:51:37,709 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:51:37,709 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 01:51:37,709 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-3-shard-people-testLeadershipTransferOnShutdown is initialized 01:51:37,709 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:51:37,709 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:51:37,709 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 01:51:37,709 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:37,709 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:51:37,709 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:37,709 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onShutDown 01:51:37,709 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Initiating leader transfer 01:51:37,709 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:51:37,709 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:51:37,709 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 01:51:37,709 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:37,709 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:51:37,709 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@40cd390e 01:51:37,709 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 01:51:37,709 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-cars-testLeadershipTransferOnShutdown is initialized 01:51:37,709 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:37,709 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@40cd390e 01:51:37,710 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 01:51:37,710 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-people-testLeadershipTransferOnShutdown from null to Follower 01:51:37,710 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:37,710 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 01:51:37,710 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:37,710 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:51:37,710 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:37,710 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5816286 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@70e00511 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} previous state SEALED 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@70e00511 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], modifications=2, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], modifications=2, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2239/0x00000008016abcb8@cb08ed2 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@49d3509 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], modifications=2, protocol=THREE_PHASE}, enqueuedTicks=852039671675} 01:51:37,710 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from null to Follower 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-632: slice: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], modifications=2, protocol=THREE_PHASE}} 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-632: Message size: 401 bytes, total slices to send: 1 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-632: Message does not need to be sliced - sending original message 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@70e00511 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} 01:51:37,710 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2ac9d9eb finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@52380706 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@56feef70 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@7f4abe1c, splicing queue 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@56feef70 queue spliced 0 messages 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 946.4 μs 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-2045441789], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-2045441789], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@38ac4f16 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@62801f7 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} previous state FLUSHED 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@62801f7 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=OPEN} 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=OPEN} 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2239/0x00000008016abcb8@d1c1b8e 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-2045441789], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@48c12ed0 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=852039867070} 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-632: slice: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], modifications=1, protocol=THREE_PHASE}} 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-632: Message size: 390 bytes, total slices to send: 1 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-632: Message does not need to be sliced - sending original message 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2ac9d9eb finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-2045441789], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@69b532bd 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7d411e98 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@5dac6408, splicing queue 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7d411e98 queue spliced 0 messages 01:51:37,711 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-2045441789], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 376.1 μs 01:51:37,712 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown] 01:51:37,712 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done true 01:51:37,712 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 01:51:37,712 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 01:51:37,712 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 01:51:37,712 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:37,712 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:51:37,712 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:37,712 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:37,713 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testLeadershipTransferOnShutdown, matchIndex: -1, lastIndex: -1, isVoting: true 01:51:37,713 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 01:51:37,713 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:37,713 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:37,713 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:51:37,713 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 01:51:37,713 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:37,713 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: not currently active leader, rejecting request Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#2090313808], modifications=2, protocol=THREE_PHASE}}. isLeader: true, isLeaderActive: false,isLeadershipTransferInProgress: true, paused: true 01:51:37,713 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:37,713 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@49d3509 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@2637ee1f, splicing queue 01:51:37,713 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:37,713 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@49d3509 queue spliced 1 messages 01:51:37,713 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:37,713 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: refreshing backend for shard 1 01:51:37,713 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:37,713 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:37,713 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:37,714 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1350683196] 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: request Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#2090313808], modifications=2, protocol=THREE_PHASE}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:51:37,714 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown] 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 01:51:37,714 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done false 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Scheduling 2000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testLeadershipTransferOnShutdown: allocating new open transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 ready 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 5 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: transitioned member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 to ready 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 initiating canCommit 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 validated 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 completed canCommit 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:51:37,714 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 01:51:37,715 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:51:37,715 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:51:37,715 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:51:37,715 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 01:51:37,715 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:37,715 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:37,715 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done true 01:51:37,715 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:51:37,716 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 01:51:37,716 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 01:51:37,716 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 01:51:37,716 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 01:51:37,717 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:37,717 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 01:51:37,717 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:37,717 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 4 - updating term 01:51:37,717 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 01:51:37,717 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:37,717 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 01:51:37,717 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:51:37,717 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:37,717 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 01:51:37,717 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:37,717 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than leader's term 4 - switching to Follower 01:51:37,717 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 01:51:37,717 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 01:51:37,717 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:37,717 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 01:51:37,717 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:51:37,717 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 5 01:51:37,717 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-636: Closing 01:51:37,718 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:51:37,718 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 01:51:37,718 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown: Closing 01:51:37,718 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Leader to Follower 01:51:37,718 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 01:51:37,718 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:51:37,718 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Leader to Follower 01:51:37,718 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:51:37,718 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 01:51:37,718 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:37,718 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:37,718 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:51:37,718 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 01:51:37,718 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [] 01:51:37,718 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0}} 01:51:37,718 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=852039867070, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0}} 01:51:37,718 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} canCommit completed 01:51:37,718 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 01:51:37,719 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:51:37,719 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 01:51:37,719 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:37,719 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 01:51:37,719 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is ready - notifying 1 OnShardInitialized callbacks 01:51:37,719 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6a3fa7de] 01:51:37,719 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:51:37,719 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:37,719 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6a3fa7de] 01:51:37,719 PM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 01:51:37,719 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:51:37,719 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6147ce9a, attempting to connect 01:51:37,719 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done false 01:51:37,720 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown: Closing 01:51:37,720 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 01:51:37,720 PM [cluster-test-shard-dispatcher-27] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-2-shard-cars-testLeadershipTransferOnShutdown in 10.30 ms 01:51:37,720 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 01:51:37,720 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} is not yet known 01:51:37,720 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:51:37,720 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:37,720 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:37,720 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:51:37,720 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-1-shard-cars-testLeadershipTransferOnShutdown status sync done false 01:51:37,720 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=9201, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:51:37,720 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:51:37,720 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:37,720 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:37,720 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 01:51:37,720 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:37,720 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:51:37,720 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:37,720 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 01:51:37,720 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:51:37,720 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:37,720 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:51:37,720 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@70e00511 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5cc1a1f 01:51:37,720 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} previous state SEALED 01:51:37,720 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} previous state SEALED 01:51:37,720 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5cc1a1f 01:51:37,720 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:51:37,720 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 01:51:37,720 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], modifications=2, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 01:51:37,720 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], modifications=2, protocol=THREE_PHASE} 01:51:37,721 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 01:51:37,721 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:37,721 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} allocated sequence 0 01:51:37,721 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2239/0x00000008016abcb8@cb08ed2 01:51:37,721 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:51:37,721 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@71c47599 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], coordinated=true}, enqueuedTicks=852039671675} 01:51:37,721 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} 01:51:37,721 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5cc1a1f 01:51:37,721 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testLeadershipTransferOnShutdown: allocating new ready transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:51:37,721 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:51:37,721 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} 01:51:37,721 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} 01:51:37,721 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], modifications=1} 01:51:37,721 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 initiating canCommit 01:51:37,721 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:51:37,721 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 validated 01:51:37,721 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:37,721 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2ac9d9eb finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:37,721 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 completed canCommit 01:51:37,721 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 01:51:37,721 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} 01:51:37,721 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@256cb5ee 01:51:37,721 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@77d66467 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@fb918b4, splicing queue 01:51:37,721 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@77d66467 queue spliced 0 messages 01:51:37,721 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 1.070 ms 01:51:37,721 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], coordinated=true} with Envelope{sessionId=2, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 01:51:37,721 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], coordinated=true}, enqueuedTicks=852039671675, sessionId=2, txSequence=0} with Envelope{sessionId=2, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 01:51:37,721 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 01:51:37,721 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:51:37,721 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@51bd9afb 01:51:37,721 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 01:51:37,722 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:51:37,722 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x0000000801303580@431fb7c1 01:51:37,722 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:37,722 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:37,722 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} 01:51:37,722 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} allocated sequence 1 01:51:37,722 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x0000000801303580@431fb7c1 01:51:37,722 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:37,722 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@71c47599 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2090313808]}, enqueuedTicks=852052696596} 01:51:37,722 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 01:51:37,722 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@1ddf3f37 01:51:37,722 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} 01:51:37,722 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} allocated sequence 1 01:51:37,722 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x0000000801303580@30da43c7 01:51:37,722 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-2045441789], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:37,722 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@48c12ed0 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2090313808]}, enqueuedTicks=852052815859} 01:51:37,722 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 initiating preCommit 01:51:37,722 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:51:37,722 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 candidate ready 01:51:37,722 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1: canCommit - candidate rootPath: / 01:51:37,722 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1: canCommit - messages: [] 01:51:37,722 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 01:51:37,722 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 prepared 01:51:37,722 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:37,722 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 completed preCommit 01:51:37,722 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} with Envelope{sessionId=2, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1}} 01:51:37,722 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2090313808]}, enqueuedTicks=852052696596, sessionId=2, txSequence=1} with Envelope{sessionId=2, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1}} 01:51:37,722 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:51:37,729 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 initiating preCommit 01:51:37,729 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:51:37,729 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 candidate ready 01:51:37,729 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2: canCommit - candidate rootPath: / 01:51:37,729 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2: canCommit - messages: [] 01:51:37,729 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2: preCommit - successfulFromPrevious: [] 01:51:37,729 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 prepared 01:51:37,729 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:37,729 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 completed preCommit 01:51:37,730 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1}} 01:51:37,730 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2090313808]}, enqueuedTicks=852052815859, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1}} 01:51:37,730 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:51:37,730 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:51:37,730 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@331262ba 01:51:37,730 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 01:51:37,730 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:51:37,730 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@331262ba 01:51:37,730 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:37,730 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:37,730 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} 01:51:37,730 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} allocated sequence 2 01:51:37,730 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@331262ba 01:51:37,730 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:37,730 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@71c47599 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2090313808]}, enqueuedTicks=852061302229} 01:51:37,730 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 2 01:51:37,730 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@57d0d568 01:51:37,730 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} 01:51:37,730 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} allocated sequence 2 01:51:37,730 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@57d0d568 01:51:37,730 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-2045441789], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:37,730 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@48c12ed0 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2090313808]}, enqueuedTicks=852061447742} 01:51:37,731 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 initiating commit 01:51:37,731 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:51:37,731 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 170 01:51:37,731 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 01:51:37,731 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 01:51:37,731 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:37,731 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:37,731 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,731 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 initiating commit 01:51:37,731 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}]]] 01:51:37,731 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:51:37,731 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:37,731 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:37,731 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-1-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}]]] 01:51:37,731 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 submitted to persistence 01:51:37,732 PM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 174 01:51:37,732 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:51:37,732 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:51:37,732 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:37,732 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:37,732 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 01:51:37,732 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 01:51:37,732 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:37,732 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 submitted to persistence 01:51:37,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 01:51:37,732 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}]]] 01:51:37,733 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:51:37,733 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:37,733 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=170} 01:51:37,733 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 01:51:37,733 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 1 01:51:37,733 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:37,733 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:37,733 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 01:51:37,733 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 01:51:37,733 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-3-shard-people-testLeadershipTransferOnShutdown: Closing 01:51:37,733 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 01:51:37,733 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 01:51:37,733 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done true 01:51:37,734 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 01:51:37,734 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:51:37,734 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:37,734 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=174} 01:51:37,734 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:51:37,734 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 1 01:51:37,734 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:37,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:51:37,734 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:51:37,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 01:51:37,734 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 01:51:37,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:51:37,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 0 01:51:37,735 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:37,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:51:37,735 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 01:51:37,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=170} 01:51:37,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:51:37,735 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 01:51:37,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@221b4a9}, purged=MutableUnsignedLongSet{size=0}} 01:51:37,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 01:51:37,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:51:37,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1: commit - successfulFromPrevious: [] 01:51:37,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:51:37,735 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 01:51:37,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1468630134] with name member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 01:51:37,735 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:51:37,735 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Setting last applied to 0 01:51:37,735 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:51:37,735 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=174} 01:51:37,735 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:51:37,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:37,735 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-1-shard-people-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@22070628}, purged=MutableUnsignedLongSet{size=0}} 01:51:37,735 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 01:51:37,735 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} with Envelope{sessionId=2, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2}} 01:51:37,735 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:51:37,735 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:51:37,735 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2: commit - successfulFromPrevious: [] 01:51:37,735 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2090313808]}, enqueuedTicks=852061302229, sessionId=2, txSequence=2} with Envelope{sessionId=2, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2}} 01:51:37,735 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 01:51:37,735 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 01:51:37,735 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5816286 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 01:51:37,735 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 01:51:37,735 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 01:51:37,735 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@24afb7cd 01:51:37,735 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 01:51:37,735 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1320043685] with name member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 01:51:37,735 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 01:51:37,735 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 01:51:37,735 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@642cb573 01:51:37,736 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:37,736 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1093849851], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:37,736 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} 01:51:37,736 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED}: initiating purge 01:51:37,736 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=DONE} allocated sequence 3 01:51:37,736 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:37,736 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@209d10cd 01:51:37,736 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:37,736 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:51:37,736 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@71c47599 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2090313808]}, enqueuedTicks=852066368634} 01:51:37,736 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1468630134] 01:51:37,736 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:51:37,736 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 01:51:37,736 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:51:37,736 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 01:51:37,736 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:37,736 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 134.8 μs 01:51:37,736 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1468630134] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:37,736 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1320043685] 01:51:37,736 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:51:37,736 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 43.32 μs 01:51:37,736 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1320043685] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:37,737 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2}} 01:51:37,737 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#2090313808]}, enqueuedTicks=852061447742, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2}} 01:51:37,737 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} doCommit completed 01:51:37,737 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@38ac4f16 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} 01:51:37,737 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR}: initiating purge 01:51:37,737 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 3 01:51:37,737 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@6575136d 01:51:37,737 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} 01:51:37,737 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED}: initiating purge 01:51:37,737 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=DONE} allocated sequence 3 01:51:37,737 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@5eab33d8 01:51:37,737 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-2045441789], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:37,737 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@48c12ed0 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2090313808]}, enqueuedTicks=852067953113} 01:51:37,737 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=FLUSHED} allocated sequence 0 01:51:37,737 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@53f6d5b0 01:51:37,737 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@71c47599 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], coordinated=false}, enqueuedTicks=852068458115} 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testLeadershipTransferOnShutdown: allocating new ready transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 initiating direct canCommit 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 validated 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 initiating direct preCommit 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 candidate ready 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1: canCommit - candidate rootPath: / 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1: canCommit - messages: [] 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 prepared 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 initiating direct commit 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 229 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:51:37,738 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,739 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=5, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}], SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}]]] 01:51:37,739 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:51:37,739 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:37,739 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-1-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}], SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}], SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}]]] 01:51:37,739 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 submitted to persistence 01:51:37,739 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 01:51:37,739 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 01:51:37,739 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:51:37,739 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 01:51:37,739 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:51:37,739 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=174} 01:51:37,739 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:51:37,739 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 01:51:37,739 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-people-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4ec5eb8c}, purged=MutableUnsignedLongSet{size=0}} 01:51:37,739 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 01:51:37,740 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:51:37,740 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-830193633] with name member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 01:51:37,740 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-830193633] 01:51:37,740 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:51:37,740 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:51:37,740 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-830193633] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:37,740 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 64.98 μs 01:51:37,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:51:37,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 01:51:37,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 01:51:37,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:37,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 01:51:37,741 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:51:37,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 01:51:37,741 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:37,741 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:37,741 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:37,741 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:51:37,741 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 01:51:37,741 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:51:37,741 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 01:51:37,741 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 01:51:37,741 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-people-testLeadershipTransferOnShutdown status sync done false 01:51:37,741 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:51:37,741 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:51:37,741 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:37,741 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=170} 01:51:37,741 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=174} 01:51:37,741 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:51:37,742 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 1 01:51:37,742 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:37,742 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:51:37,742 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:51:37,742 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-people-testLeadershipTransferOnShutdown status sync done true 01:51:37,742 PM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendMetadata - member-3-shard-cars-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5b76f2ba}, purged=MutableUnsignedLongSet{size=0}} 01:51:37,742 PM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 01:51:37,742 PM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:51:37,742 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 01:51:37,742 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#1366207773] with name member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 01:51:37,742 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#1366207773] 01:51:37,742 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=5, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}], SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}]]] 01:51:37,742 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 2 01:51:37,742 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:51:37,742 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62} 01:51:37,742 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 01:51:37,742 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=229} 01:51:37,742 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 01:51:37,742 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:51:37,742 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 3 01:51:37,742 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:37,742 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 01:51:37,742 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:51:37,742 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 01:51:37,742 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 01:51:37,742 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 73.64 μs 01:51:37,743 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 01:51:37,743 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 01:51:37,743 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:51:37,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 01:51:37,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 01:51:37,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 1 01:51:37,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:51:37,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62} 01:51:37,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: replication of member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@5976a137 01:51:37,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testLeadershipTransferOnShutdown: finished purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:51:37,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 01:51:37,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:51:37,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 2 01:51:37,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 01:51:37,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=229} 01:51:37,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:51:37,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 01:51:37,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:51:37,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1: commit - successfulFromPrevious: [] 01:51:37,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 01:51:37,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:51:37,743 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62} 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 2 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testLeadershipTransferOnShutdown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:37,744 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:51:37,744 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} with Envelope{sessionId=2, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3}} 01:51:37,744 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2090313808]}, enqueuedTicks=852066368634, sessionId=2, txSequence=3} with Envelope{sessionId=2, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3}} 01:51:37,744 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=DONE}: purge completed 01:51:37,744 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 52.87 μs 01:51:37,744 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5cc1a1f purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=DONE} 01:51:37,744 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:51:37,744 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@70e00511 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 01:51:37,744 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:51:37,744 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5816286 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 01:51:37,744 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], coordinated=false} with Envelope{sessionId=2, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0}} 01:51:37,744 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808], coordinated=false}, enqueuedTicks=852068458115, sessionId=2, txSequence=4} with Envelope{sessionId=2, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0}} 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 01:51:37,744 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=FLUSHED} directCommit completed 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:51:37,744 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=FLUSHED}: initiating purge 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:51:37,744 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=DONE} allocated sequence 1 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62} 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 01:51:37,744 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@256499bf 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 2 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-testLeadershipTransferOnShutdown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:37,744 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:51:37,744 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@71c47599 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2090313808]}, enqueuedTicks=852074952406} 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=174} 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-people-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-3-shard-people-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@25ed9ad0}, purged=MutableUnsignedLongSet{size=0}} 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-people-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:51:37,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1686486481] with name member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 01:51:37,744 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1686486481] 01:51:37,744 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:51:37,745 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 29.68 μs 01:51:37,745 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 01:51:37,745 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 01:51:37,745 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2ac9d9eb: creating a new snapshot member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 01:51:37,745 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 01:51:37,745 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@394b36fe: creating a new snapshot member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:51:37,745 PM [main] [DEBUG] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 01:51:37,745 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:37,745 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:37,745 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1875823032] 01:51:37,745 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} shard 1 01:51:37,745 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 01:51:37,745 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:37,745 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1548096848], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@3422e277 01:51:37,745 PM [main] [DEBUG] AbstractClientConnection - member-3-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:51:37,745 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:37,745 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 01:51:37,745 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 01:51:37,745 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 01:51:37,745 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:51:37,745 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Setting last applied to 1 01:51:37,745 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:51:37,745 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@2e2a731b, attempting to connect 01:51:37,745 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62} 01:51:37,745 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: replication of member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@545a9ad1 01:51:37,746 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testLeadershipTransferOnShutdown: finished purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:51:37,746 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 01:51:37,746 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:51:37,746 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 01:51:37,746 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:37,746 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 01:51:37,746 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 2 01:51:37,746 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 01:51:37,746 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 01:51:37,746 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:51:37,746 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62} 01:51:37,746 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 01:51:37,746 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:51:37,746 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 2 01:51:37,746 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 01:51:37,746 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=229} 01:51:37,746 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:51:37,746 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 01:51:37,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:37,747 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 01:51:37,747 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:51:37,747 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 01:51:37,747 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 01:51:37,747 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 42.28 μs 01:51:37,747 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3}} 01:51:37,747 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#2090313808]}, enqueuedTicks=852067953113, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3}} 01:51:37,747 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=DONE}: purge completed 01:51:37,747 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@62801f7 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=DONE} 01:51:37,747 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR}: purge completed 01:51:37,747 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@38ac4f16 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} 01:51:37,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 01:51:37,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 1 01:51:37,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:51:37,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 01:51:37,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:51:37,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62} 01:51:37,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 01:51:37,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:51:37,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 01:51:37,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 01:51:37,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:51:37,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 01:51:37,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 01:51:37,748 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:37,748 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} is not yet known 01:51:37,748 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=4980, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:51:37,748 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 3 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,748 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}]]] 01:51:37,750 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:37,750 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 01:51:37,750 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}]]] 01:51:37,750 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 01:51:37,750 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:51:37,750 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63} 01:51:37,750 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 01:51:37,750 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 4 01:51:37,750 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=3, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:37,750 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 1 01:51:37,750 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:51:37,750 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 01:51:37,750 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 01:51:37,750 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62} 01:51:37,750 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 01:51:37,750 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-people-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 01:51:37,750 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 01:51:37,750 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 01:51:37,751 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 3, nextIndex: 4 01:51:37,751 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 01:51:37,751 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 3 01:51:37,751 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 2 to 3 01:51:37,751 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63} 01:51:37,751 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: replication of member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@16ae363b 01:51:37,751 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testLeadershipTransferOnShutdown: finished purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:51:37,751 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 01:51:37,751 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:51:37,751 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} with Envelope{sessionId=2, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1}} 01:51:37,751 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:51:37,751 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2090313808]}, enqueuedTicks=852074952406, sessionId=2, txSequence=5} with Envelope{sessionId=2, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1}} 01:51:37,751 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=DONE}: purge completed 01:51:37,751 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=3 01:51:37,751 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5cc1a1f purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=DONE} 01:51:37,753 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 01:51:37,753 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:37,753 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 3 01:51:37,753 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 01:51:37,753 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 3 01:51:37,753 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 2 to 3 01:51:37,753 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:37,753 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63} 01:51:37,753 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 01:51:37,753 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 01:51:37,753 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@111713a9 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@73fa5220 01:51:37,753 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=3 01:51:37,753 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:51:37,753 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@73fa5220 01:51:37,753 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:51:37,753 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 01:51:37,753 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1548096848], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 01:51:37,753 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:37,753 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1548096848], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1247/0x0000000801301400@412936f2 01:51:37,753 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-3-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:37,753 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@43b1986c transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1548096848], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=852076199656} 01:51:37,753 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@394b36fe finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:37,753 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 01:51:37,753 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@62fbb968 01:51:37,753 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@76fc5862 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@7e6a7ee9, splicing queue 01:51:37,753 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@76fc5862 queue spliced 0 messages 01:51:37,753 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 534.8 μs 01:51:37,754 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testLeadershipTransferOnShutdown: allocating new open snapshot member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 01:51:37,754 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-2-shard-cars-testLeadershipTransferOnShutdown_slicer-id-633: slice: identifier: member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 01:51:37,754 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-2-shard-cars-testLeadershipTransferOnShutdown_slicer-id-633: Message size: 363 bytes, total slices to send: 1 01:51:37,754 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-2-shard-cars-testLeadershipTransferOnShutdown_slicer-id-633: Message does not need to be sliced - sending original message 01:51:37,755 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1548096848], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 01:51:37,755 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1548096848], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=852076199656, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 01:51:37,755 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0} 01:51:37,755 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:51:37,761 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,761 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-3-shard-people-testLeadershipTransferOnShutdown 01:51:37,761 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onShutDown 01:51:37,761 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 01:51:37,761 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onShutDown 01:51:37,761 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-3-shard-cars-testLeadershipTransferOnShutdown 01:51:37,761 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 01:51:37,761 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Stopping Shard member-3-shard-people-testLeadershipTransferOnShutdown 01:51:37,761 PM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 01:51:37,762 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:51:37,762 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:51:37,762 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 01:51:37,762 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 01:51:37,762 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:51:37,762 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:51:37,762 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-1-shard-cars-testLeadershipTransferOnShutdown 01:51:37,762 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 01:51:37,762 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-3-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:51:37,762 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-1-shard-people-testLeadershipTransferOnShutdown 01:51:37,762 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 01:51:37,762 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onShutDown 01:51:37,763 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Initiating leader transfer 01:51:37,763 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:51:37,763 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:37,763 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:37,763 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2c81423f 01:51:37,763 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2c81423f 01:51:37,763 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 01:51:37,763 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:37,763 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:37,764 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown] 01:51:37,764 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 01:51:37,764 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:37,764 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 01:51:37,764 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@48c12ed0 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@34465b12, splicing queue 01:51:37,764 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@48c12ed0 queue spliced 0 messages 01:51:37,764 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-2045441789], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-2045441789], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:37,764 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: refreshing backend for shard 2 01:51:37,765 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-2045441789], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:51:37,765 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:51:37,765 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:51:37,765 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testLeadershipTransferOnShutdown, matchIndex: 1, lastIndex: 1, isVoting: true 01:51:37,765 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 01:51:37,765 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1350683196] 01:51:37,765 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:37,765 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:37,765 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:51:37,765 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 01:51:37,765 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Scheduling 2000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:51:37,765 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 01:51:37,765 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 01:51:37,765 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 01:51:37,765 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 5 01:51:37,765 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-people-testLeadershipTransferOnShutdown 01:51:37,766 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] to peer member-3-shard-people-testLeadershipTransferOnShutdown 01:51:37,766 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:51:37,766 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 01:51:37,766 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:51:37,766 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:51:37,766 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown: Closing 01:51:37,766 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 01:51:37,766 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 01:51:37,766 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 01:51:37,766 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 01:51:37,766 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 01:51:37,771 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:37,771 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 01:51:37,771 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 01:51:37,771 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4]" message is greater than leader's term 4 - switching to Follower 01:51:37,771 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 01:51:37,771 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 01:51:37,771 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 01:51:37,771 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 01:51:37,771 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:51:37,771 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 5 01:51:37,771 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-people-testLeadershipTransferOnShutdown (Leader)_slicer-id-635: Closing 01:51:37,771 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 01:51:37,771 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 01:51:37,771 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown: Closing 01:51:37,772 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Leader to Follower 01:51:37,772 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 01:51:37,772 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Leader to Follower 01:51:37,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:51:37,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:51:37,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 01:51:37,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:37,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:37,772 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:51:37,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 01:51:37,772 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 01:51:37,772 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 01:51:37,772 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:37,772 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 01:51:37,773 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: became leader with frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 01:51:37,773 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 01:51:37,773 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Candidate to Leader 01:51:37,773 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is ready - notifying 1 OnShardInitialized callbacks 01:51:37,773 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for people: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1454008181, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@214c0857] 01:51:37,773 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 01:51:37,773 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1454008181, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@214c0857] 01:51:37,773 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@27002638, attempting to connect 01:51:37,773 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown: Closing 01:51:37,773 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 01:51:37,773 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:51:37,773 PM [cluster-test-shard-dispatcher-26] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-2-shard-people-testLeadershipTransferOnShutdown in 10.70 ms 01:51:37,773 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 01:51:37,773 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-1-shard-people-testLeadershipTransferOnShutdown status sync done false 01:51:37,774 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-1-shard-people-testLeadershipTransferOnShutdown 01:51:37,774 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 01:51:37,774 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1454008181], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present} 01:51:37,774 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-2045441789], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1454008181], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 01:51:37,774 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@62801f7 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@15e147b4 01:51:37,774 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:51:37,774 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2ac9d9eb finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1454008181], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 01:51:37,774 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 01:51:37,774 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@13f900d1 01:51:37,774 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@56380e29 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1ba56026, splicing queue 01:51:37,774 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@56380e29 queue spliced 0 messages 01:51:37,774 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:37,774 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#-2045441789], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1454008181], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} in 316.7 μs 01:51:37,774 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 01:51:37,774 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 01:51:37,774 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:51:37,774 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 01:51:37,774 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 01:51:37,775 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:51:37,775 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:51:37,775 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:37,775 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:37,778 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:37,779 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:37,779 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:37,779 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:37,780 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:51:37,789 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLeadershipTransferOnShutdown: Closing 01:51:37,789 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-cars-testLeadershipTransferOnShutdown 01:51:37,789 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-640: Closing 01:51:37,789 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:51:37,789 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-people-testLeadershipTransferOnShutdown 01:51:37,789 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-2-shard-people-testLeadershipTransferOnShutdown (Leader)_slicer-id-641: Closing 01:51:37,789 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 01:51:37,789 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 01:51:37,791 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:37,791 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:38,788 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [6123523105792681763] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:38,807 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:38,807 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:38,809 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:51:38,809 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [6628146650783934260] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [03/11/2024 13:51:38.810] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:38,816 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:38,816 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/11/2024 13:51:38.819] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:38,837 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:38,842 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [6737073753841734414] 01:51:38,843 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:51:38,844 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:51:38,844 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:51:38,851 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:38,851 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:38,853 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:51:38,853 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:51:38,853 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:51:38,856 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:38,863 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [8816591792776387560] 01:51:38,863 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:51:38,864 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:38,864 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:51:38,874 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:38,874 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:38,886 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:38,894 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [3701435524437765158] 01:51:38,894 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:51:38,895 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:38,895 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:51:38,902 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:38,902 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:38,905 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:38,906 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:38,907 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionForwardedToLeaderAfterRetry 01:51:38,914 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 01:51:38,917 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1340100173] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:38,923 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionForwardedToLeaderAfterRetry 01:51:38,923 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:38,923 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:38,923 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:38,923 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:38,923 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:38,925 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: no initial generation override, starting from 0 01:51:38,925 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:51:38,925 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@73be5ac6 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@f63197a 01:51:38,925 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@b5c8c4e 01:51:38,925 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, 0, 1710165098925, None)) 01:51:38,926 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165098924,0,0)) 01:51:38,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5b0805c7 01:51:38,926 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@f63197a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4a0dbf36 01:51:38,926 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} started 01:51:38,926 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:38,928 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:38,929 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionForwardedToLeaderAfterRetry 01:51:38,930 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:38,930 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:38,930 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:38,930 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:38,930 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076] 01:51:38,930 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:38,931 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:38,931 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:38,931 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874] 01:51:38,931 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, persistent : true 01:51:38,931 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, persistent : true 01:51:38,931 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 01:51:38,931 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:38,931 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:38,931 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-1534903727 created and ready for shard:member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:38,931 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-1534903727] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:38,931 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 01:51:38,931 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#1072520105] 01:51:38,931 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#364983113 created and ready for shard:member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:38,931 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#1072520105] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:38,932 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#143752908] 01:51:38,933 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:38,933 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:38,933 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:38,933 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:38,933 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:51:38,933 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:51:38,933 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:51:38,933 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:51:38,933 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [cars] 01:51:38,933 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is initialized 01:51:38,933 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 01:51:38,933 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [people] 01:51:38,933 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 01:51:38,933 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is initialized 01:51:38,933 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 01:51:38,933 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 01:51:38,933 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from null to Follower 01:51:38,933 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from null to Follower 01:51:38,934 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#364983113] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:38,934 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#143752908] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:38,934 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:51:38,934 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:51:38,936 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:38,936 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:38,940 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 01:51:38,941 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1625961809] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:38,942 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionForwardedToLeaderAfterRetry 01:51:38,942 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:38,942 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:38,942 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:38,942 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:38,942 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:38,942 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: no initial generation override, starting from 0 01:51:38,942 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:51:38,942 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4f259377 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@38b44afb 01:51:38,942 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7db6c389 01:51:38,942 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, 0, 1710165098942, None)) 01:51:38,942 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165098941,0,0)) 01:51:38,943 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@38b44afb to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@33dc5f52 01:51:38,943 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} started 01:51:38,943 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7dddf3bb 01:51:38,943 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:38,943 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:38,943 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:38,943 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:38,943 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031] 01:51:38,943 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:38,944 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, persistent : true 01:51:38,944 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 01:51:38,944 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:38,944 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:38,944 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034] 01:51:38,944 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, persistent : true 01:51:38,944 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:38,944 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:38,944 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#800045525] 01:51:38,944 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 01:51:38,944 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#312710329 created and ready for shard:member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:38,944 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-1670442773 created and ready for shard:member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:38,944 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-1113782737] 01:51:38,944 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#312710329] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:38,945 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:38,945 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:38,945 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:51:38,945 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:51:38,945 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [cars] 01:51:38,945 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:38,945 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is initialized 01:51:38,945 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:38,945 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:38,945 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 01:51:38,945 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:51:38,945 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 01:51:38,945 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:51:38,945 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:38,945 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 01:51:38,946 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#800045525] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:38,946 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-1670442773] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:38,946 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-1113782737] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:38,946 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [people] 01:51:38,946 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is initialized 01:51:38,946 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from null to Follower 01:51:38,946 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 01:51:38,946 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from null to Follower 01:51:38,993 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1340100173] 01:51:38,993 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:51:38,993 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076] 01:51:39,175 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:39,175 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Received ElectionTimeout - switching to Candidate 01:51:39,175 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 01:51:39,175 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=1, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:39,175 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 1 01:51:39,176 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:39,176 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:51:39,176 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:51:39,176 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:51:39,176 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 01:51:39,176 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 01:51:39,205 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:39,205 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Received ElectionTimeout - switching to Candidate 01:51:39,205 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 01:51:39,205 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:39,205 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 1 01:51:39,205 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:39,205 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:51:39,205 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:51:39,205 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:51:39,205 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 01:51:39,205 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 01:51:39,405 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:51:39,405 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=2, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:39,405 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 2 01:51:39,475 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:51:39,475 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:39,475 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 2 01:51:39,555 PM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:39,565 PM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:39,674 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:51:39,674 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:39,674 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 3 01:51:39,705 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:51:39,705 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:39,705 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 3 01:51:39,865 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:39,865 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:39,866 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:39,866 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:39,866 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:39,866 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:39,866 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:39,866 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:39,866 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:39,866 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:39,866 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:39,866 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:39,866 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:51:39,866 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:51:39,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:51:39,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:39,915 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 4 01:51:39,915 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:39,916 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:39,916 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 01:51:39,916 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:39,916 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:39,916 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=null 01:51:39,916 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:39,916 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:39,916 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:39,923 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:39,923 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:39,924 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 01:51:39,924 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:39,924 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:39,924 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 01:51:39,924 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [] 01:51:39,924 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 01:51:39,924 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-1-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:39,925 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:51:39,925 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:39,925 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:39,925 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done false 01:51:39,925 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: -1, nextIndex: 0 01:51:39,925 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:39,926 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 01:51:39,926 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:51:39,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:39,926 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:39,926 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 01:51:39,984 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 01:51:39,984 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:39,984 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 4 01:51:39,984 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:39,986 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:39,986 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 01:51:39,986 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:39,986 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:39,986 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=null 01:51:39,986 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:39,986 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:39,986 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:39,987 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:39,987 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:39,987 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 01:51:39,987 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:39,987 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:39,987 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 01:51:39,987 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [] 01:51:39,987 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 01:51:39,987 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:51:39,987 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:39,987 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:39,987 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 01:51:39,987 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 01:51:39,988 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 01:51:39,988 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:39,988 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done false 01:51:39,989 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: -1, nextIndex: 0 01:51:39,989 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:39,989 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:51:39,989 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:39,989 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:39,989 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 01:51:40,35 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done true 01:51:40,49 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1340100173] 01:51:40,49 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 01:51:40,49 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874] 01:51:40,49 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1340100173] 01:51:40,49 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:51:40,49 PM [cluster-test-akka.actor.default-dispatcher-33] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076] 01:51:40,49 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076] 01:51:40,49 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1625961809] 01:51:40,49 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:51:40,49 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031] 01:51:40,50 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031] 01:51:40,50 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21877e70: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:51:40,50 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 01:51:40,50 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:40,50 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:40,50 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1625961809] 01:51:40,50 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:40,50 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} shard 1 01:51:40,50 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:51:40,50 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:51:40,50 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:40,50 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 01:51:40,50 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:51:40,50 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:51:40,50 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1625961809] 01:51:40,50 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:51:40,50 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} shard 2 01:51:40,50 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:51:40,50 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:51:40,50 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:51:40,51 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:51:40,51 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21877e70 readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:51:40,51 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:51:40,51 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@6ea1babd 01:51:40,51 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:51:40,51 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@6eba2fe8 01:51:40,51 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} scheduling timeout in 30 seconds 01:51:40,51 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@5fa49ffe, attempting to connect 01:51:40,51 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@2abf3e5e, attempting to connect 01:51:40,52 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 01:51:40,52 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=17810, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:51:40,52 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 01:51:40,52 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=6160, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:51:40,53 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:40,53 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:40,53 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:51:40,54 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:40,54 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4c06c286 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3cf0e6a9 01:51:40,54 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} previous state FLUSHED 01:51:40,54 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3cf0e6a9 01:51:40,54 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:51:40,54 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} 01:51:40,54 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} 01:51:40,54 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:51:40,54 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2239/0x00000008016abcb8@2702f403 01:51:40,54 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:51:40,54 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@14ee12e9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=854381730505} 01:51:40,54 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE}} 01:51:40,54 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: Message size: 399 bytes, total slices to send: 1 01:51:40,54 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: Message does not need to be sliced - sending original message 01:51:40,54 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21877e70 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@4aabe53b 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@52388500 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@5fd9b992, splicing queue 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@52388500 queue spliced 0 messages 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 1.092 ms 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@be2a71c instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3de9889b 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3de9889b 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2239/0x00000008016abcb8@39e83375 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@605becf8 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=854381664074} 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, message: Envelope{sessionId=1, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE}} 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: Message size: 396 bytes, total slices to send: 1 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: Message does not need to be sliced - sending original message 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21877e70 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 01:51:40,55 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@b8813ec 01:51:40,56 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3c5d30cf superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1f4133a4, splicing queue 01:51:40,56 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3c5d30cf queue spliced 0 messages 01:51:40,56 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 707.2 μs 01:51:40,59 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:51:40,59 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:51:40,59 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 ready 01:51:40,60 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transitioned member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 to ready 01:51:40,60 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 initiating canCommit 01:51:40,60 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:51:40,60 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 validated 01:51:40,60 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:40,60 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 completed canCommit 01:51:40,60 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:51:40,60 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:51:40,60 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 ready 01:51:40,60 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transitioned member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 to ready 01:51:40,60 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 initiating canCommit 01:51:40,60 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:51:40,60 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 validated 01:51:40,60 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:40,60 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 completed canCommit 01:51:40,60 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 01:51:40,60 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=854381730505, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 01:51:40,60 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} canCommit completed 01:51:40,61 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 01:51:40,61 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=854381664074, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 01:51:40,61 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 01:51:40,61 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:51:40,61 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@54e28e32 01:51:40,61 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 01:51:40,61 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} allocated sequence 1 01:51:40,61 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x0000000801303580@3775b241 01:51:40,61 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:40,61 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@605becf8 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=854392107821} 01:51:40,61 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 01:51:40,61 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@80f1e28 01:51:40,61 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 01:51:40,61 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} allocated sequence 1 01:51:40,61 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x0000000801303580@a4d55f8 01:51:40,61 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:40,61 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@14ee12e9 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=854392231034} 01:51:40,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 initiating preCommit 01:51:40,62 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 initiating preCommit 01:51:40,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:51:40,62 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:51:40,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 candidate ready 01:51:40,62 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 candidate ready 01:51:40,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1: canCommit - candidate rootPath: / 01:51:40,62 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2: canCommit - candidate rootPath: / 01:51:40,62 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2: canCommit - messages: [] 01:51:40,62 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2: preCommit - successfulFromPrevious: [] 01:51:40,62 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 prepared 01:51:40,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1: canCommit - messages: [] 01:51:40,62 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:40,62 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 completed preCommit 01:51:40,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 01:51:40,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 prepared 01:51:40,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:40,62 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 completed preCommit 01:51:40,62 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1}} 01:51:40,63 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=854392231034, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1}} 01:51:40,63 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:51:40,63 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1}} 01:51:40,63 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=854392107821, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1}} 01:51:40,63 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:51:40,63 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:51:40,63 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@50639d25 01:51:40,63 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 01:51:40,63 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} allocated sequence 2 01:51:40,63 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@50639d25 01:51:40,63 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:40,63 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@605becf8 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=854393734912} 01:51:40,63 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 2 01:51:40,63 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@206d375f 01:51:40,63 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 01:51:40,63 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} allocated sequence 2 01:51:40,63 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@206d375f 01:51:40,63 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:40,63 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@14ee12e9 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=854393831354} 01:51:40,63 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 initiating commit 01:51:40,63 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:51:40,63 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 initiating commit 01:51:40,63 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:51:40,63 PM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 184 01:51:40,63 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 01:51:40,63 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 01:51:40,63 PM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 180 01:51:40,63 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:40,64 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:51:40,64 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:51:40,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:40,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:40,64 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:40,64 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:40,64 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:40,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}]]] 01:51:40,64 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 01:51:40,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 submitted to persistence 01:51:40,64 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 submitted to persistence 01:51:40,64 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 01:51:40,64 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:51:40,64 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}]]] 01:51:40,64 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 01:51:40,64 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:51:40,64 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:51:40,64 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:40,64 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:40,64 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=184} 01:51:40,64 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=180} 01:51:40,64 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 01:51:40,64 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:51:40,64 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 1 01:51:40,65 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 1 01:51:40,65 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:40,65 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:40,65 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 01:51:40,65 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:51:40,65 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done true 01:51:40,65 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 0, nextIndex: 1 01:51:40,65 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:51:40,65 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 0 01:51:40,65 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 01:51:40,65 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data Simple{size=184} 01:51:40,65 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:51:40,65 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 0, nextIndex: 1 01:51:40,65 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@9f64eba}, purged=MutableUnsignedLongSet{size=0}} 01:51:40,65 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:51:40,65 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:51:40,65 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:51:40,65 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 0 01:51:40,65 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 01:51:40,65 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2: commit - successfulFromPrevious: [] 01:51:40,65 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 01:51:40,65 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data Simple{size=180} 01:51:40,65 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:51:40,65 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1088f3b3}, purged=MutableUnsignedLongSet{size=0}} 01:51:40,65 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-741407296] with name member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 01:51:40,65 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:51:40,66 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:51:40,66 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1: commit - successfulFromPrevious: [] 01:51:40,66 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:51:40,66 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:40,66 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 01:51:40,66 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#822393211] with name member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 01:51:40,66 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#822393211] 01:51:40,66 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:40,66 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:51:40,66 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-741407296] 01:51:40,66 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 01:51:40,66 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#822393211] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:40,66 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:51:40,66 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-741407296] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:40,66 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 72.79 μs 01:51:40,66 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 59.57 μs 01:51:40,67 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 0 01:51:40,67 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:51:40,67 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 0 01:51:40,67 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 01:51:40,67 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data Simple{size=184} 01:51:40,67 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:51:40,67 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@436591bc}, purged=MutableUnsignedLongSet{size=0}} 01:51:40,67 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:51:40,67 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:51:40,67 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1990194145] with name member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 01:51:40,67 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 01:51:40,67 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1990194145] 01:51:40,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 0 01:51:40,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:51:40,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 0 01:51:40,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 01:51:40,67 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:51:40,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data Simple{size=180} 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 86.20 μs 01:51:40,68 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2}} 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=854393831354, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2}} 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} doCommit completed 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1990194145] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4c06c286 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR}: initiating purge 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 3 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@a92382f 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED}: initiating purge 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=DONE} allocated sequence 3 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@6442a522 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@14ee12e9 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=854398678714} 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2}} 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=854393734912, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2}} 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@be2a71c completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@adef350 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED}: initiating purge 01:51:40,68 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@41f622a2}, purged=MutableUnsignedLongSet{size=0}} 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=DONE} allocated sequence 3 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@1196f164 01:51:40,68 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:51:40,68 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@605becf8 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=854399082544} 01:51:40,68 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1935206740] with name member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1935206740] 01:51:40,68 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:51:40,68 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1625961809] 01:51:40,68 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 01:51:40,69 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 77.25 μs 01:51:40,69 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:51:40,69 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031] 01:51:40,69 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1935206740] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:40,69 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031] 01:51:40,69 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:51:40,69 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 01:51:40,69 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 01:51:40,69 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:51:40,69 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:40,69 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 01:51:40,69 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 01:51:40,70 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:40,70 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 01:51:40,70 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 01:51:40,119 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031] 01:51:40,169 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031] 01:51:40,220 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031] 01:51:40,224 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:40,224 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}]]] 01:51:40,225 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}]]] 01:51:40,225 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:51:40,225 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:51:40,225 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72} 01:51:40,225 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 01:51:40,225 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 01:51:40,225 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:40,226 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 01:51:40,226 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 01:51:40,226 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:51:40,226 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 1 01:51:40,226 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 01:51:40,226 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72} 01:51:40,226 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@325e337 01:51:40,226 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:51:40,226 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:51:40,227 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:51:40,227 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 01:51:40,227 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:40,227 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 01:51:40,227 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3}} 01:51:40,227 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=854399082544, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3}} 01:51:40,227 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=DONE}: purge completed 01:51:40,227 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3de9889b purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=DONE} 01:51:40,227 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:51:40,228 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@be2a71c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 01:51:40,228 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 1 01:51:40,228 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:51:40,228 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 1 01:51:40,228 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 01:51:40,228 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72} 01:51:40,228 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:51:40,228 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:51:40,228 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 01:51:40,228 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 01:51:40,270 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031] 01:51:40,271 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1625961809] 01:51:40,271 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 01:51:40,271 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034] 01:51:40,271 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034] 01:51:40,277 PM [cluster-test-shard-dispatcher-12] [WARN] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 207, lastApplied : 0, commitIndex : 0 01:51:40,277 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:40,277 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}]]] 01:51:40,277 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}]]] 01:51:40,277 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:51:40,277 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:51:40,277 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72} 01:51:40,277 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 01:51:40,278 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 01:51:40,278 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:40,278 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 01:51:40,278 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 01:51:40,278 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:51:40,278 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 1 01:51:40,278 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 01:51:40,278 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72} 01:51:40,278 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@3eac2976 01:51:40,278 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:51:40,278 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:51:40,278 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:51:40,278 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 01:51:40,278 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:40,278 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 01:51:40,279 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3}} 01:51:40,279 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=854398678714, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3}} 01:51:40,279 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=DONE}: purge completed 01:51:40,279 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3cf0e6a9 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=DONE} 01:51:40,279 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR}: purge completed 01:51:40,279 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4c06c286 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 01:51:40,279 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 1 01:51:40,279 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:51:40,279 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 1 01:51:40,279 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 01:51:40,279 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72} 01:51:40,279 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:51:40,279 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:51:40,280 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 01:51:40,280 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 01:51:40,322 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034] 01:51:40,322 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21877e70: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:51:40,322 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:51:40,322 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:51:40,322 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:51:40,322 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=OPEN} allocated sequence 0 01:51:40,322 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21877e70 readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 01:51:40,322 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@432d6cbe 01:51:40,322 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:40,322 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@605becf8 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=854653427252} 01:51:40,323 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, message: Envelope{sessionId=1, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE}} 01:51:40,323 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: Message size: 407 bytes, total slices to send: 1 01:51:40,323 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: Message does not need to be sliced - sending original message 01:51:40,323 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@d723495 01:51:40,323 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:40,323 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@14ee12e9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=854653892253} 01:51:40,323 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, message: Envelope{sessionId=0, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE}} 01:51:40,323 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: Message size: 401 bytes, total slices to send: 1 01:51:40,323 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: Message does not need to be sliced - sending original message 01:51:40,325 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:51:40,326 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:51:40,326 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:51:40,326 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 ready 01:51:40,326 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transitioned member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 to ready 01:51:40,326 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 initiating canCommit 01:51:40,326 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:51:40,326 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 validated 01:51:40,326 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:40,326 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 completed canCommit 01:51:40,326 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:51:40,326 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 ready 01:51:40,326 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transitioned member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 to ready 01:51:40,326 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 initiating canCommit 01:51:40,326 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:51:40,326 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 validated 01:51:40,326 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:40,326 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 completed canCommit 01:51:40,328 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=4, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0}} 01:51:40,328 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=854653892253, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0}} 01:51:40,328 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=FLUSHED} canCommit completed 01:51:40,328 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=4, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0}} 01:51:40,328 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=854653427252, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0}} 01:51:40,328 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=FLUSHED} canCommit completed 01:51:40,328 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21877e70: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:51:40,328 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:51:40,328 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=OPEN} allocated sequence 0 01:51:40,329 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:51:40,329 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=OPEN} allocated sequence 0 01:51:40,329 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21877e70 readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 01:51:40,329 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1340100173] 01:51:40,329 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:51:40,329 PM [cluster-test-akka.actor.default-dispatcher-33] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076] 01:51:40,329 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076] 01:51:40,329 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1625961809] 01:51:40,329 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:51:40,329 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031] 01:51:40,329 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031] 01:51:40,329 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21877e70: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:51:40,330 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:51:40,330 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} allocated sequence 0 01:51:40,330 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=2} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@2412c8ea 01:51:40,330 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:40,330 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@605becf8 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=2}, enqueuedTicks=854660793434} 01:51:40,330 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, message: Envelope{sessionId=1, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=2}} 01:51:40,330 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: Message size: 491 bytes, total slices to send: 1 01:51:40,330 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: Message does not need to be sliced - sending original message 01:51:40,330 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} allocated sequence 1 01:51:40,330 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=2} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@47f9f4ff 01:51:40,330 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@605becf8 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=2}, enqueuedTicks=854661472071} 01:51:40,330 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, message: Envelope{sessionId=1, txSequence=6, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=2}} 01:51:40,331 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: Message size: 492 bytes, total slices to send: 1 01:51:40,331 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: Message does not need to be sliced - sending original message 01:51:40,331 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} allocated sequence 2 01:51:40,331 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21877e70: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 01:51:40,331 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 01:51:40,331 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=OPEN} allocated sequence 0 01:51:40,331 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21877e70: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:51:40,331 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:51:40,331 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=OPEN} allocated sequence 0 01:51:40,331 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1340100173] 01:51:40,331 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:51:40,331 PM [cluster-test-akka.actor.default-dispatcher-33] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076] 01:51:40,331 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076] 01:51:40,332 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:51:40,335 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=2} with Envelope{sessionId=1, txSequence=5, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0}} 01:51:40,335 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=2}, enqueuedTicks=854660793434, sessionId=1, txSequence=5} with Envelope{sessionId=1, txSequence=5, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0}} 01:51:40,335 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=2} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0} 01:51:40,335 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=2} with Envelope{sessionId=1, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1}} 01:51:40,335 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=2}, enqueuedTicks=854661472071, sessionId=1, txSequence=6} with Envelope{sessionId=1, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1}} 01:51:40,335 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=2} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1} 01:51:40,382 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076] 01:51:40,382 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1625961809] 01:51:40,382 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:51:40,382 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031] 01:51:40,382 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031] 01:51:40,382 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionForwardedToLeaderAfterRetry 01:51:40,383 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:40,383 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:40,383 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config null, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:40,383 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config null, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:40,383 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-648: Closing 01:51:40,383 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:51:40,383 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-649: Closing 01:51:40,383 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:51:40,383 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:51:40,383 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:51:40,383 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: removing frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 01:51:40,383 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 01:51:40,383 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: removing frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 01:51:40,383 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 01:51:40,384 PM [cluster-test-akka.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Leader to Follower 01:51:40,384 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Leader to Follower 01:51:40,384 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 01:51:40,384 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:40,384 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:40,384 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 01:51:40,384 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:40,384 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:40,384 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Leader to Follower 01:51:40,384 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Leader to Follower 01:51:40,433 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1340100173] 01:51:40,433 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:51:40,433 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076] 01:51:40,433 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@1d2ec644 01:51:40,433 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:40,433 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@605becf8 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=854764160325} 01:51:40,433 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, message: Envelope{sessionId=1, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE}} 01:51:40,433 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: Message size: 453 bytes, total slices to send: 1 01:51:40,433 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: Message does not need to be sliced - sending original message 01:51:40,434 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@47a13e24 01:51:40,434 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:40,434 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@14ee12e9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=854764568695} 01:51:40,434 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, message: Envelope{sessionId=0, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE}} 01:51:40,434 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: Message size: 434 bytes, total slices to send: 1 01:51:40,434 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-645: Message does not need to be sliced - sending original message 01:51:40,434 PM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:51:40,434 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21877e70 readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 01:51:40,434 PM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 01:51:40,434 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21877e70 readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 01:51:40,434 PM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:51:40,434 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21877e70 readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 01:51:40,434 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionForwardedToLeaderAfterRetry 01:51:40,435 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:40,435 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:40,435 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 01:51:40,435 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:40,435 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 01:51:40,435 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Re-initializing to Follower with previous leaderId member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:40,435 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:51:40,435 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:51:40,435 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : true 01:51:40,435 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : true 01:51:40,435 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:51:40,435 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:51:40,437 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: not currently active leader, rejecting request Envelope{sessionId=1, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1354758025], modifications=1, protocol=THREE_PHASE}}. isLeader: false, isLeaderActive: true,isLeadershipTransferInProgress: false, paused: false 01:51:40,437 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: request Envelope{sessionId=1, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1354758025], modifications=1, protocol=THREE_PHASE}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:51:40,437 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: not currently active leader, rejecting request Envelope{sessionId=0, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1354758025], modifications=1, protocol=THREE_PHASE}}. isLeader: false, isLeaderActive: true,isLeadershipTransferInProgress: false, paused: false 01:51:40,437 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: request Envelope{sessionId=0, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1354758025], modifications=1, protocol=THREE_PHASE}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:51:40,442 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} indicated no leadership, reconnecting it org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:51:40,442 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@605becf8 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@186c583f, splicing queue 01:51:40,442 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:40,442 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@605becf8 queue spliced 1 messages 01:51:40,442 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:40,442 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: refreshing backend for shard 1 01:51:40,443 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:40,443 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:40,443 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:40,443 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1625961809] 01:51:40,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:40,443 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} indicated no leadership, reconnecting it org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:51:40,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:51:40,443 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@14ee12e9 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@778cf31f, splicing queue 01:51:40,443 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:51:40,443 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:51:40,443 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@14ee12e9 queue spliced 1 messages 01:51:40,443 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:40,443 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: refreshing backend for shard 2 01:51:40,443 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:51:40,443 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@61c5ec2d, attempting to connect 01:51:40,443 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:51:40,443 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:51:40,443 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1625961809] 01:51:40,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:51:40,443 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:51:40,443 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:51:40,444 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@2305b48, attempting to connect 01:51:40,445 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 01:51:40,445 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: not currently leader, rejecting request ConnectClientRequest{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry$c], minVersion=POTASSIUM, maxVersion=POTASSIUM}. isLeader: false, isLeaderActive: true,isLeadershipTransferInProgress: false. 01:51:40,445 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 01:51:40,445 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: not currently leader, rejecting request ConnectClientRequest{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-people-testTransactionForwardedToLeaderAfterRetry$d], minVersion=POTASSIUM, maxVersion=POTASSIUM}. isLeader: false, isLeaderActive: true,isLeadershipTransferInProgress: false. 01:51:40,447 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Connect attempt to cars failed, will retry org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:51:40,447 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:40,447 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:40,447 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:40,447 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Connect attempt to people failed, will retry org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:51:40,448 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 2 shard people failed, removing state java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:40,448 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timed out resolving shard 2, scheduling retry in 1 second java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:40,448 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:40,485 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1625961809] 01:51:40,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:51:40,485 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031] 01:51:40,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Received TimeoutNow - switching to Candidate 01:51:40,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry] 01:51:40,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:40,485 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 5 01:51:40,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:40,485 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:51:40,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:51:40,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:51:40,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:51:40,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 01:51:40,485 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 01:51:40,485 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 01:51:40,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:40,485 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 01:51:40,486 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:40,486 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:40,486 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 01:51:40,486 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 01:51:40,486 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 4 - updating term 01:51:40,486 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=null 01:51:40,486 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 01:51:40,486 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:40,486 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:51:40,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:51:40,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:51:40,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry] 01:51:40,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:40,488 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:51:40,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 01:51:40,488 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:51:40,488 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:40,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 01:51:40,488 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 01:51:40,488 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:40,488 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 01:51:40,488 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 01:51:40,489 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 01:51:40,489 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:40,489 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done false 01:51:40,489 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:51:40,489 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:40,489 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:40,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 01:51:40,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:40,496 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done true 01:51:40,535 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1625961809] 01:51:40,535 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 01:51:40,535 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034] 01:51:40,535 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=FLUSHED} allocated sequence 1 01:51:40,535 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=FLUSHED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@3c4114fe 01:51:40,535 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Received TimeoutNow - switching to Candidate 01:51:40,535 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:40,535 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testTransactionForwardedToLeaderAfterRetry] 01:51:40,535 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:40,535 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=FLUSHED} allocated sequence 1 01:51:40,535 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=FLUSHED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@5d7c0fe5 01:51:40,535 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 5 01:51:40,535 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:40,535 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:51:40,535 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3cf0e6a9 skipping transactions [3, 4, 5] 01:51:40,535 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:51:40,535 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 01:51:40,536 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 01:51:40,536 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 01:51:40,536 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:40,536 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 01:51:40,536 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:40,536 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 01:51:40,536 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:40,536 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 01:51:40,536 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 01:51:40,536 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:40,536 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 01:51:40,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 01:51:40,537 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 4 - updating term 01:51:40,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=null 01:51:40,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 01:51:40,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:40,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:51:40,541 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:51:40,542 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:51:40,542 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-1-shard-people-testTransactionForwardedToLeaderAfterRetry] 01:51:40,542 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:40,542 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:51:40,542 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 01:51:40,542 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:51:40,542 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 01:51:40,542 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:40,542 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 01:51:40,542 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 01:51:40,542 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 01:51:40,542 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 01:51:40,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 01:51:40,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:40,543 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done false 01:51:40,543 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 01:51:40,543 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:40,543 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:40,543 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 01:51:40,543 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 01:51:40,543 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:40,549 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done true 01:51:40,607 PM [cluster-test-shard-dispatcher-31] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 110, lastApplied : 1, commitIndex : 1 01:51:40,656 PM [cluster-test-shard-dispatcher-31] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 105, lastApplied : 1, commitIndex : 1 01:51:40,726 PM [cluster-test-shard-dispatcher-31] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 01:51:40,781 PM [cluster-test-shard-dispatcher-31] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 124, lastApplied : 1, commitIndex : 1 01:51:40,846 PM [cluster-test-shard-dispatcher-31] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 01:51:40,885 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:51:40,895 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:40,895 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:40,895 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:40,895 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:40,895 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:40,895 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 01:51:40,895 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:40,895 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:40,895 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:40,895 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:40,895 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:40,895 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 01:51:40,898 PM [cluster-test-shard-dispatcher-27] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 115, lastApplied : 1, commitIndex : 1 01:51:40,928 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:51:40,966 PM [cluster-test-shard-dispatcher-27] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 01:51:41,16 PM [cluster-test-shard-dispatcher-27] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 117, lastApplied : 1, commitIndex : 1 01:51:41,86 PM [cluster-test-shard-dispatcher-27] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 114, lastApplied : 1, commitIndex : 1 01:51:41,136 PM [cluster-test-shard-dispatcher-27] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 119, lastApplied : 1, commitIndex : 1 01:51:41,207 PM [cluster-test-shard-dispatcher-27] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 119, lastApplied : 1, commitIndex : 1 01:51:41,256 PM [cluster-test-shard-dispatcher-27] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 01:51:41,326 PM [cluster-test-shard-dispatcher-27] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 01:51:41,375 PM [cluster-test-shard-dispatcher-27] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 117, lastApplied : 1, commitIndex : 1 01:51:41,446 PM [cluster-test-shard-dispatcher-27] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 01:51:41,465 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:41,465 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:41,465 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:41,465 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1625961809] 01:51:41,465 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 2 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:41,465 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:51:41,465 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:51:41,465 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1625961809] 01:51:41,465 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:41,465 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@374077ed] 01:51:41,465 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:51:41,465 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@62f27551] 01:51:41,465 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@374077ed] 01:51:41,466 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@62f27551] 01:51:41,466 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@1859af1c, attempting to connect 01:51:41,466 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@27a4d8e5, attempting to connect 01:51:41,466 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:51:41,466 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:51:41,466 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:51:41,466 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:41,466 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3de9889b instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@359f9238 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} previous state FLUSHED 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} previous state FLUSHED 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} previous state SEALED 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} previous state SEALED 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} previous state SEALED 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@359f9238 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=OPEN} 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Forwarding successful request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=OPEN} 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} allocated sequence 0 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1246/0x0000000801301c40@75de0da8 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@74a29803 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=true}, enqueuedTicks=854764160325} 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 replayed 1 successful requests 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Replaying queued request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} allocated sequence 1 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} enqueing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@3c4114fe 01:51:41,467 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@74a29803 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=854866247355} 01:51:41,467 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@359f9238 01:51:41,467 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 initiating canCommit 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=OPEN} 01:51:41,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=OPEN} 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} 01:51:41,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 validated 01:51:41,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} allocated sequence 0 01:51:41,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 completed canCommit 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@1d2ec644 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@74a29803 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=true}, enqueuedTicks=854764160325} 01:51:41,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 initiating preCommit 01:51:41,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@359f9238 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Forwarding successful request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=2} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=2} 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Forwarding successful request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=2} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} 01:51:41,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 candidate ready 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=2} 01:51:41,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1: canCommit - candidate rootPath: / 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 replayed 2 successful requests 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} 01:51:41,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1} 01:51:41,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1: canCommit - messages: [] 01:51:41,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 01:51:41,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 prepared 01:51:41,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@359f9238 01:51:41,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 completed preCommit 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=OPEN} 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=OPEN} 01:51:41,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1} 01:51:41,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 initiating canCommit 01:51:41,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@359f9238 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=OPEN} 01:51:41,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 validated 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=OPEN} 01:51:41,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1} 01:51:41,468 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 completed canCommit 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21877e70 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} 01:51:41,468 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@10c39f78 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@693b68d2 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3ebadb00, splicing queue 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@693b68d2 queue spliced 0 messages 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 2.180 ms 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present} 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3cf0e6a9 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@811f337 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} previous state FLUSHED 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} previous state FLUSHED 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@811f337 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=OPEN} 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Forwarding successful request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=OPEN} 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} allocated sequence 0 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1246/0x0000000801301c40@75de0da8 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} scheduling timeout in 30 seconds 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@36ea10bc transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=true}, enqueuedTicks=854764568695} 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 replayed 1 successful requests 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Replaying queued request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} allocated sequence 1 01:51:41,469 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} enqueing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@5d7c0fe5 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@36ea10bc transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=854866660056} 01:51:41,469 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 initiating canCommit 01:51:41,469 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@811f337 01:51:41,469 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 validated 01:51:41,469 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=OPEN} 01:51:41,469 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 completed canCommit 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=OPEN} 01:51:41,469 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], modifications=1, protocol=THREE_PHASE} 01:51:41,469 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 initiating preCommit 01:51:41,469 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:51:41,470 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} allocated sequence 0 01:51:41,470 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@47a13e24 01:51:41,470 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@36ea10bc transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=true}, enqueuedTicks=854764568695} 01:51:41,470 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 candidate ready 01:51:41,470 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2: canCommit - candidate rootPath: / 01:51:41,470 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21877e70 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 01:51:41,470 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} 01:51:41,470 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2: canCommit - messages: [] 01:51:41,470 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} 01:51:41,470 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2: preCommit - successfulFromPrevious: [] 01:51:41,470 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@59ef671a 01:51:41,470 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 prepared 01:51:41,470 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:41,470 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@714ae430 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@17c37555, splicing queue 01:51:41,470 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 completed preCommit 01:51:41,470 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@714ae430 queue spliced 0 messages 01:51:41,470 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:51:41,470 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} in 1.113 ms 01:51:41,470 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 initiating canCommit 01:51:41,470 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:51:41,470 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 validated 01:51:41,470 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=true} with Envelope{sessionId=2, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0}} 01:51:41,470 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:41,470 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=true}, enqueuedTicks=854764160325, sessionId=2, txSequence=0} with Envelope{sessionId=2, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0}} 01:51:41,470 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 completed canCommit 01:51:41,470 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} with Envelope{sessionId=2, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1}} 01:51:41,470 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=854866247355, sessionId=2, txSequence=1} with Envelope{sessionId=2, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1}} 01:51:41,470 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:51:41,470 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=true} with Envelope{sessionId=2, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0}} 01:51:41,470 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=true}, enqueuedTicks=854764160325, sessionId=2, txSequence=2} with Envelope{sessionId=2, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0}} 01:51:41,470 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} canCommit completed 01:51:41,471 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=true} with Envelope{sessionId=3, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0}} 01:51:41,471 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=true}, enqueuedTicks=854764568695, sessionId=3, txSequence=0} with Envelope{sessionId=3, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0}} 01:51:41,471 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} with Envelope{sessionId=3, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1}} 01:51:41,471 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=854866660056, sessionId=3, txSequence=1} with Envelope{sessionId=3, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1}} 01:51:41,471 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:51:41,471 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=true} with Envelope{sessionId=3, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0}} 01:51:41,471 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=true}, enqueuedTicks=854764568695, sessionId=3, txSequence=2} with Envelope{sessionId=3, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0}} 01:51:41,471 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} canCommit completed 01:51:41,471 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 2 01:51:41,471 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@44ee56f1 01:51:41,471 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:41,471 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:41,471 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} 01:51:41,471 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} allocated sequence 2 01:51:41,471 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@44ee56f1 01:51:41,471 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:41,471 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@74a29803 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855802026794} 01:51:41,471 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} allocated sequence 2 01:51:41,471 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@703c4c76 01:51:41,471 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:41,471 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:41,471 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 initiating commit 01:51:41,471 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} 01:51:41,471 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:51:41,471 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} allocated sequence 2 01:51:41,471 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@703c4c76 01:51:41,471 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 01:51:41,471 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@36ea10bc transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855802193938} 01:51:41,471 PM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 78 01:51:41,471 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 01:51:41,471 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 01:51:41,471 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:41,471 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:51:41,471 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 initiating commit 01:51:41,471 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 2 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:41,471 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:51:41,471 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}]]] 01:51:41,471 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 submitted to persistence 01:51:41,472 PM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 185 01:51:41,472 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}] 01:51:41,472 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}] 01:51:41,472 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:41,472 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:51:41,472 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 2 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:41,472 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 01:51:41,472 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}]]] 01:51:41,472 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 submitted to persistence 01:51:41,473 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}] 01:51:41,474 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}]]] 01:51:41,474 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:51:41,474 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:51:41,474 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=78} 01:51:41,474 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 01:51:41,474 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 01:51:41,474 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}]]] 01:51:41,474 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:41,474 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:51:41,474 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:51:41,474 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=185} 01:51:41,474 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}] 01:51:41,474 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 01:51:41,474 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:41,474 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}] 01:51:41,474 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 01:51:41,475 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 2, nextIndex: 3 01:51:41,475 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:51:41,475 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 2 01:51:41,475 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 01:51:41,475 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=185} 01:51:41,475 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:51:41,475 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:51:41,475 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:51:41,475 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2: commit - successfulFromPrevious: [] 01:51:41,475 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 changing state from COMMIT_PENDING to COMMITTED 01:51:41,475 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} with Envelope{sessionId=3, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2}} 01:51:41,475 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855802193938, sessionId=3, txSequence=3} with Envelope{sessionId=3, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2}} 01:51:41,475 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} doCommit completed 01:51:41,475 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3cf0e6a9 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} 01:51:41,475 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR}: initiating purge 01:51:41,475 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} allocated sequence 3 01:51:41,475 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@6ef07c55 01:51:41,475 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 01:51:41,475 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:41,475 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:41,475 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} 01:51:41,475 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED}: initiating purge 01:51:41,475 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=DONE} allocated sequence 3 01:51:41,475 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:51:41,475 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@48bd7edf 01:51:41,475 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 01:51:41,475 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@36ea10bc transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855806344781} 01:51:41,475 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 01:51:41,475 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:51:41,476 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:51:41,476 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 70.39 μs 01:51:41,476 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 01:51:41,476 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 01:51:41,476 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:41,476 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 2, nextIndex: 3 01:51:41,476 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:51:41,476 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 2 01:51:41,476 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 01:51:41,476 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=78} 01:51:41,476 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:51:41,476 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:51:41,476 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:51:41,476 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1: commit - successfulFromPrevious: [] 01:51:41,476 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 01:51:41,476 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} with Envelope{sessionId=2, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2}} 01:51:41,476 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855802026794, sessionId=2, txSequence=3} with Envelope{sessionId=2, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2}} 01:51:41,476 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 01:51:41,476 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} doCommit completed 01:51:41,476 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3de9889b completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} 01:51:41,476 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR}: initiating purge 01:51:41,476 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 3 01:51:41,476 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@2092a066 01:51:41,476 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:51:41,476 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:41,476 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:41,476 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} 01:51:41,476 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED}: initiating purge 01:51:41,476 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 01:51:41,476 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=DONE} allocated sequence 3 01:51:41,476 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@4d6d72f5 01:51:41,476 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:41,476 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@74a29803 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855807189182} 01:51:41,476 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:51:41,476 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} allocated sequence 1 01:51:41,476 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@51ab73c4 01:51:41,476 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:41,476 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:41,476 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} 01:51:41,476 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} allocated sequence 1 01:51:41,476 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@51ab73c4 01:51:41,476 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@74a29803 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855807484919} 01:51:41,476 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 35.20 μs 01:51:41,476 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} allocated sequence 1 01:51:41,476 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@6744eede 01:51:41,476 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:41,477 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:41,477 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} 01:51:41,477 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} allocated sequence 1 01:51:41,477 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@6744eede 01:51:41,477 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@36ea10bc transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855807648023} 01:51:41,478 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:51:41,478 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 01:51:41,478 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 01:51:41,478 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:41,478 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 initiating preCommit 01:51:41,478 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:51:41,478 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 candidate ready 01:51:41,478 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1: canCommit - candidate rootPath: / 01:51:41,478 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1: canCommit - messages: [] 01:51:41,478 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1: preCommit - successfulFromPrevious: [] 01:51:41,478 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 prepared 01:51:41,478 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 initiating preCommit 01:51:41,478 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:41,478 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:51:41,478 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 completed preCommit 01:51:41,478 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 candidate ready 01:51:41,478 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2: canCommit - candidate rootPath: / 01:51:41,478 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} with Envelope{sessionId=2, txSequence=5, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1}} 01:51:41,478 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855807484919, sessionId=2, txSequence=5} with Envelope{sessionId=2, txSequence=5, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1}} 01:51:41,478 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2: canCommit - messages: [] 01:51:41,478 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2: preCommit - successfulFromPrevious: [] 01:51:41,478 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 prepared 01:51:41,478 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:51:41,478 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:41,478 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 completed preCommit 01:51:41,479 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} with Envelope{sessionId=3, txSequence=5, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1}} 01:51:41,479 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855807648023, sessionId=3, txSequence=5} with Envelope{sessionId=3, txSequence=5, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1}} 01:51:41,479 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:51:41,479 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} allocated sequence 2 01:51:41,479 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@2b601640 01:51:41,479 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 01:51:41,479 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:41,479 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:41,479 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} 01:51:41,479 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} allocated sequence 2 01:51:41,479 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@2b601640 01:51:41,479 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@74a29803 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855809874108} 01:51:41,479 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 01:51:41,479 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} allocated sequence 2 01:51:41,479 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@21df65bb 01:51:41,479 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:41,479 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:41,479 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 initiating commit 01:51:41,479 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} 01:51:41,479 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:51:41,479 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} allocated sequence 2 01:51:41,479 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@21df65bb 01:51:41,479 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@36ea10bc transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855810036272} 01:51:41,479 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 initiating commit 01:51:41,479 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:51:41,479 PM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 237 01:51:41,479 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 01:51:41,479 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 01:51:41,479 PM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 223 01:51:41,479 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}] 01:51:41,479 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}] 01:51:41,479 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:41,479 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:41,479 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 01:51:41,479 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:41,479 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 01:51:41,479 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:41,479 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}], SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}]]] 01:51:41,479 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 submitted to persistence 01:51:41,479 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}], SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}]]] 01:51:41,479 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 submitted to persistence 01:51:41,480 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}] 01:51:41,480 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 01:51:41,480 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 2 01:51:41,480 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 2 01:51:41,480 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:51:41,480 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:51:41,480 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 2 01:51:41,480 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 01:51:41,480 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 2 01:51:41,480 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 01:51:41,480 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=78} 01:51:41,480 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=185} 01:51:41,480 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:51:41,480 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:51:41,480 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:51:41,480 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:51:41,480 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 01:51:41,480 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 01:51:41,480 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:51:41,480 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:51:41,480 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:51:41,480 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 01:51:41,481 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 31.42 μs 01:51:41,481 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 01:51:41,481 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:51:41,481 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 34.30 μs 01:51:41,481 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}], SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}]]] 01:51:41,481 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 01:51:41,481 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:51:41,481 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73} 01:51:41,481 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 01:51:41,481 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=223} 01:51:41,481 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}], SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}]]] 01:51:41,481 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}] 01:51:41,481 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 01:51:41,481 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:51:41,481 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 01:51:41,481 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73} 01:51:41,481 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 01:51:41,481 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=4, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:41,481 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=237} 01:51:41,481 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 01:51:41,481 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 01:51:41,481 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=4, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:41,482 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 01:51:41,482 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}] 01:51:41,482 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:41,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:41,482 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 01:51:41,482 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 01:51:41,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 4, nextIndex: 5 01:51:41,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 2 01:51:41,482 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 4, nextIndex: 5 01:51:41,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 3 01:51:41,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 01:51:41,482 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 2 01:51:41,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73} 01:51:41,482 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 3 01:51:41,482 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 01:51:41,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@a77d0a7 01:51:41,482 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73} 01:51:41,482 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@78d5ca3 01:51:41,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:51:41,482 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:51:41,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:51:41,482 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} with Envelope{sessionId=3, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3}} 01:51:41,482 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:51:41,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:51:41,482 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855806344781, sessionId=3, txSequence=4} with Envelope{sessionId=3, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3}} 01:51:41,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 4 01:51:41,482 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=DONE}: purge completed 01:51:41,482 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:51:41,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 01:51:41,482 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@811f337 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=DONE} 01:51:41,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=223} 01:51:41,482 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR}: purge completed 01:51:41,482 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 4 01:51:41,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:51:41,482 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3cf0e6a9 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} 01:51:41,482 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 01:51:41,482 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=237} 01:51:41,482 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} with Envelope{sessionId=2, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3}} 01:51:41,482 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:51:41,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:51:41,482 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855807189182, sessionId=2, txSequence=4} with Envelope{sessionId=2, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3}} 01:51:41,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:51:41,482 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=DONE}: purge completed 01:51:41,482 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:51:41,482 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@359f9238 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=DONE} 01:51:41,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2: commit - successfulFromPrevious: [] 01:51:41,482 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR}: purge completed 01:51:41,482 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:51:41,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 changing state from COMMIT_PENDING to COMMITTED 01:51:41,482 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3de9889b purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} 01:51:41,482 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1: commit - successfulFromPrevious: [] 01:51:41,482 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 changing state from COMMIT_PENDING to COMMITTED 01:51:41,482 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} with Envelope{sessionId=3, txSequence=6, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2}} 01:51:41,482 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855810036272, sessionId=3, txSequence=6} with Envelope{sessionId=3, txSequence=6, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2}} 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} doCommit completed 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3cf0e6a9 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR}: initiating purge 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} allocated sequence 3 01:51:41,483 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@6e85b601 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:41,483 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#1684015874], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:41,483 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED}: initiating purge 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=DONE} allocated sequence 3 01:51:41,483 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@6cc52a52 01:51:41,483 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@36ea10bc transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855813571379} 01:51:41,483 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} with Envelope{sessionId=2, txSequence=6, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2}} 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855809874108, sessionId=2, txSequence=6} with Envelope{sessionId=2, txSequence=6, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2}} 01:51:41,483 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} doCommit completed 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3de9889b completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR}: initiating purge 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} allocated sequence 3 01:51:41,483 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@7fbd0d3f 01:51:41,483 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 52.41 μs 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:41,483 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED}: initiating purge 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=DONE} allocated sequence 3 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@3c4b9977 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 47.21 μs 01:51:41,483 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@74a29803 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855813877137} 01:51:41,483 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=FLUSHED} allocated sequence 0 01:51:41,483 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@5e96e468 01:51:41,483 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@74a29803 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=false}, enqueuedTicks=855814166424} 01:51:41,483 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:51:41,483 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 01:51:41,483 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 01:51:41,483 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:41,483 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:51:41,483 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 initiating direct canCommit 01:51:41,483 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:51:41,484 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 validated 01:51:41,484 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:41,484 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 initiating direct preCommit 01:51:41,484 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:51:41,484 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 01:51:41,484 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 candidate ready 01:51:41,484 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1: canCommit - candidate rootPath: / 01:51:41,484 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1: canCommit - messages: [] 01:51:41,484 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1: preCommit - successfulFromPrevious: [] 01:51:41,484 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 prepared 01:51:41,484 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:41,484 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 initiating direct commit 01:51:41,484 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:51:41,484 PM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 325 01:51:41,484 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 01:51:41,484 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 01:51:41,484 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:41,484 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 5, leaderLastIndex: 6, leaderSnapShotIndex: 3 01:51:41,484 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 5 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:41,484 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=4, prevLogTerm=5, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}], SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}]]] 01:51:41,484 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 submitted to persistence 01:51:41,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 4 01:51:41,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 01:51:41,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 3 01:51:41,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 01:51:41,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73} 01:51:41,485 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 01:51:41,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:51:41,485 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 01:51:41,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 01:51:41,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 4 01:51:41,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 01:51:41,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=237} 01:51:41,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:51:41,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:51:41,485 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:51:41,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 01:51:41,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 01:51:41,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 4 01:51:41,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 01:51:41,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 3 01:51:41,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 01:51:41,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73} 01:51:41,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:51:41,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 01:51:41,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 4 01:51:41,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 01:51:41,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=223} 01:51:41,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:51:41,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:51:41,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:51:41,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 01:51:41,486 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 01:51:41,486 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:51:41,486 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 38.92 μs 01:51:41,486 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:51:41,486 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 20.34 μs 01:51:41,487 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=4, prevLogTerm=5, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}], SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}]]] 01:51:41,487 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 01:51:41,487 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 01:51:41,487 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73} 01:51:41,487 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 01:51:41,487 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=325} 01:51:41,487 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 01:51:41,487 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 01:51:41,487 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=6, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:41,488 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 01:51:41,488 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 01:51:41,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 5 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:41,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 6, nextIndex: 7 01:51:41,488 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 5 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:41,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 4 01:51:41,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 5 01:51:41,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 01:51:41,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73} 01:51:41,488 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=4, prevLogTerm=5, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}]]] 01:51:41,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@3f29d548 01:51:41,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:51:41,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:51:41,488 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} with Envelope{sessionId=2, txSequence=7, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3}} 01:51:41,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:51:41,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 6 01:51:41,488 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855813877137, sessionId=2, txSequence=7} with Envelope{sessionId=2, txSequence=7, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3}} 01:51:41,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 5 to 6 01:51:41,488 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=DONE}: purge completed 01:51:41,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 6 data Simple{size=325} 01:51:41,488 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@359f9238 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=DONE} 01:51:41,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:51:41,488 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR}: purge completed 01:51:41,488 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3de9889b purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} 01:51:41,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:51:41,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:51:41,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1: commit - successfulFromPrevious: [] 01:51:41,488 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 changing state from COMMIT_PENDING to COMMITTED 01:51:41,489 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=false} with Envelope{sessionId=2, txSequence=8, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0}} 01:51:41,489 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=false}, enqueuedTicks=855814166424, sessionId=2, txSequence=8} with Envelope{sessionId=2, txSequence=8, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0}} 01:51:41,489 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=FLUSHED} directCommit completed 01:51:41,489 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=FLUSHED}: initiating purge 01:51:41,489 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=DONE} allocated sequence 1 01:51:41,489 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@1f7cffc9 01:51:41,489 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 5 for term 5 01:51:41,489 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@74a29803 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855819653999} 01:51:41,489 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=6 01:51:41,489 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 31.62 μs 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:51:41,489 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=FLUSHED} allocated sequence 0 01:51:41,489 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@240bf5a2 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 01:51:41,489 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@74a29803 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=false}, enqueuedTicks=855819929006} 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 initiating direct canCommit 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 validated 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 initiating direct preCommit 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 candidate ready 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1: canCommit - candidate rootPath: / 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1: canCommit - messages: [] 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1: preCommit - successfulFromPrevious: [] 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 prepared 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 initiating direct commit 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:51:41,489 PM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 237 01:51:41,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 01:51:41,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 01:51:41,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, logIndex: 8, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:41,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 7, leaderLastIndex: 8, leaderSnapShotIndex: 5 01:51:41,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 7 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:41,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=6, prevLogTerm=5, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}], SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}]]] 01:51:41,490 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 submitted to persistence 01:51:41,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 01:51:41,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 01:51:41,491 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=4, prevLogTerm=5, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}]]] 01:51:41,491 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 01:51:41,491 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 01:51:41,491 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73} 01:51:41,491 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 01:51:41,491 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 01:51:41,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 6 01:51:41,491 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=5, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:41,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 4, lastIndex(): 6 01:51:41,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 5 01:51:41,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 01:51:41,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73} 01:51:41,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:51:41,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 01:51:41,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 6 01:51:41,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 5 to 6 01:51:41,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 6 data Simple{size=325} 01:51:41,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:51:41,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:51:41,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:51:41,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 5 for term 5 01:51:41,491 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=6, prevLogTerm=5, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}], SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}]]] 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73} 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=237} 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=8, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:41,492 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:51:41,492 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 34.07 μs 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=6 01:51:41,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 7 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 5, nextIndex: 6 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 01:51:41,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 8, nextIndex: 9 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 5 01:51:41,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 8, lastAppliedIndex: 6 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 01:51:41,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 7 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73} 01:51:41,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 6 to 7 01:51:41,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 7 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73} 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@7560741e 01:51:41,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@1e5ac89e 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:51:41,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:51:41,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:51:41,492 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} with Envelope{sessionId=3, txSequence=7, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3}} 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:51:41,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:51:41,492 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855813571379, sessionId=3, txSequence=7} with Envelope{sessionId=3, txSequence=7, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3}} 01:51:41,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 8 01:51:41,492 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 4 for term 5 01:51:41,493 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=DONE}: purge completed 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 7 to 8 01:51:41,493 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@811f337 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=DONE} 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 8 data Simple{size=237} 01:51:41,493 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR}: purge completed 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:51:41,493 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3cf0e6a9 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} 01:51:41,493 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 01:51:41,493 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} with Envelope{sessionId=2, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1}} 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:51:41,493 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855819653999, sessionId=2, txSequence=9} with Envelope{sessionId=2, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1}} 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:51:41,493 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=5 01:51:41,493 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=DONE}: purge completed 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1: commit - successfulFromPrevious: [] 01:51:41,493 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@359f9238 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=DONE} 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 changing state from COMMIT_PENDING to COMMITTED 01:51:41,493 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=false} with Envelope{sessionId=2, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0}} 01:51:41,493 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:51:41,493 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=false}, enqueuedTicks=855819929006, sessionId=2, txSequence=10} with Envelope{sessionId=2, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0}} 01:51:41,493 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=FLUSHED} directCommit completed 01:51:41,493 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=FLUSHED}: initiating purge 01:51:41,493 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=DONE} allocated sequence 1 01:51:41,493 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@7919b316 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 7 for term 5 01:51:41,493 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:41,493 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@74a29803 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855823822222} 01:51:41,493 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 44.95 μs 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: 7 01:51:41,493 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=FLUSHED} allocated sequence 0 01:51:41,493 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@556add5a 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=8 01:51:41,493 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@74a29803 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=false}, enqueuedTicks=855823973826} 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, logIndex: 9, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 initiating direct canCommit 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 validated 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 initiating direct preCommit 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 candidate ready 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1: canCommit - candidate rootPath: / 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1: canCommit - messages: [] 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1: preCommit - successfulFromPrevious: [] 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 prepared 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 initiating direct commit 01:51:41,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:51:41,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 237 01:51:41,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 01:51:41,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 01:51:41,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, logIndex: 10, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:41,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 9, leaderLastIndex: 10, leaderSnapShotIndex: 7 01:51:41,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 9 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:41,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=8, prevLogTerm=5, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}], SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}]]] 01:51:41,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 submitted to persistence 01:51:41,494 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 5 01:51:41,494 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 8 01:51:41,494 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 8, context.getLastApplied(): 6, lastIndex(): 8 01:51:41,494 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5 01:51:41,494 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 7 01:51:41,494 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 5 01:51:41,494 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 6 to 7 01:51:41,494 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 01:51:41,494 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 7 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73} 01:51:41,494 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73} 01:51:41,494 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:51:41,494 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 01:51:41,494 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 01:51:41,494 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 01:51:41,494 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 8 01:51:41,494 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 7 to 8 01:51:41,494 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 4 for term 5 01:51:41,495 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 8 data Simple{size=237} 01:51:41,495 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=5 01:51:41,495 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:51:41,495 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:51:41,495 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:51:41,495 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 7 for term 5 01:51:41,495 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=8, prevLogTerm=5, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}], SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}]]] 01:51:41,495 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 01:51:41,495 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 8, entries to be added from: 0 01:51:41,495 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73} 01:51:41,495 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 01:51:41,495 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=237} 01:51:41,495 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 01:51:41,495 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 01:51:41,495 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=10, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:41,495 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:51:41,495 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 34.78 μs 01:51:41,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 9 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:41,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 10, nextIndex: 11 01:51:41,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 11, leaderLastIndex: 10, leaderSnapShotIndex: 7 01:51:41,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 01:51:41,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 9, lastAppliedIndex: 8 01:51:41,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 9 01:51:41,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 8 to 9 01:51:41,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 9 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73} 01:51:41,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@361e434 01:51:41,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:51:41,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:51:41,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 01:51:41,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 8 for term 5 01:51:41,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=9 01:51:41,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 01:51:41,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 10, lastAppliedIndex: 9 01:51:41,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 10 01:51:41,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 9 to 10 01:51:41,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 10 data Simple{size=237} 01:51:41,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:51:41,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 01:51:41,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:51:41,497 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} with Envelope{sessionId=2, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1}} 01:51:41,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1: commit - successfulFromPrevious: [] 01:51:41,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 changing state from COMMIT_PENDING to COMMITTED 01:51:41,497 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855823822222, sessionId=2, txSequence=11} with Envelope{sessionId=2, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1}} 01:51:41,497 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=DONE}: purge completed 01:51:41,497 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@359f9238 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=DONE} 01:51:41,497 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=false} with Envelope{sessionId=2, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0}} 01:51:41,497 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1354758025], coordinated=false}, enqueuedTicks=855823973826, sessionId=2, txSequence=12} with Envelope{sessionId=2, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0}} 01:51:41,497 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=FLUSHED} directCommit completed 01:51:41,497 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=FLUSHED}: initiating purge 01:51:41,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 9 for term 5 01:51:41,497 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=DONE} allocated sequence 1 01:51:41,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=10 01:51:41,497 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@4db6de9 01:51:41,497 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:41,497 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 01:51:41,497 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@74a29803 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1354758025]}, enqueuedTicks=855827841741} 01:51:41,497 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 42.48 μs 01:51:41,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 01:51:41,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 01:51:41,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 01:51:41,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, logIndex: 11, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:41,497 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1340100173] 01:51:41,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 01:51:41,497 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:51:41,497 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076] 01:51:41,498 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1882190076] 01:51:41,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 01:51:41,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 01:51:41,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=8 01:51:41,499 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1625961809] 01:51:41,499 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 01:51:41,499 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031] 01:51:41,499 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031] 01:51:41,499 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4673b99a: creating a new snapshot member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:51:41,499 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 01:51:41,499 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:41,499 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:41,499 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1340100173] 01:51:41,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:41,499 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} shard 1 01:51:41,499 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:51:41,499 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:51:41,499 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:41,499 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1992989371], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@17582e08 01:51:41,499 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:51:41,500 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:51:41,500 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@1c17d3e4, attempting to connect 01:51:41,500 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 01:51:41,501 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=8100, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:51:41,501 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:41,501 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:41,501 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:41,501 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3deb5810 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3464881 01:51:41,501 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:51:41,501 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3464881 01:51:41,501 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:51:41,501 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} 01:51:41,501 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1992989371], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} 01:51:41,502 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:41,502 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1992989371], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1247/0x0000000801301400@1d54e6bc 01:51:41,502 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:41,502 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7890e7c6 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1992989371], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=855830487927} 01:51:41,502 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4673b99a finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:41,502 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 01:51:41,502 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@4923aa15 01:51:41,502 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2dcddf83 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1716ad80, splicing queue 01:51:41,502 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2dcddf83 queue spliced 0 messages 01:51:41,502 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 403.0 μs 01:51:41,502 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new open snapshot member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 01:51:41,503 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry_slicer-id-646: slice: identifier: member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 01:51:41,503 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SlicedMessageState - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry_slicer-id-646: Message size: 514 bytes, total slices to send: 1 01:51:41,503 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry_slicer-id-646: Message does not need to be sliced - sending original message 01:51:41,504 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1992989371], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 01:51:41,504 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1992989371], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=855830487927, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 01:51:41,504 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0} 01:51:41,504 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 01:51:41,504 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:51:41,504 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:51:41,504 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1340100173] 01:51:41,504 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:51:41,504 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} shard 2 01:51:41,504 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:51:41,504 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 for transaction member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 01:51:41,504 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:51:41,504 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1992989371], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@5510482e 01:51:41,504 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} scheduling timeout in 30 seconds 01:51:41,504 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 01:51:41,505 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@3b2fd4a8, attempting to connect 01:51:41,506 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 01:51:41,506 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=4830, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:51:41,507 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:41,507 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:51:41,507 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:41,507 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6f5866ec instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3ce0ba6c 01:51:41,507 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} previous state OPEN 01:51:41,507 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3ce0ba6c 01:51:41,507 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 for transaction member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:51:41,507 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} 01:51:41,507 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1992989371], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person} to successor RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} 01:51:41,507 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:51:41,507 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} enqueing request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1992989371], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1247/0x0000000801301400@5dca837b 01:51:41,507 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:51:41,507 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@be832ff transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1992989371], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person}, enqueuedTicks=855835473320} 01:51:41,507 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4673b99a finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:41,507 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 01:51:41,507 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@32e2ca34 01:51:41,507 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5fc07837 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@5618e9ef, splicing queue 01:51:41,507 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5fc07837 queue spliced 0 messages 01:51:41,507 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 415.8 μs 01:51:41,508 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new open snapshot member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 01:51:41,508 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry_slicer-id-647: slice: identifier: member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 01:51:41,508 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SlicedMessageState - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry_slicer-id-647: Message size: 360 bytes, total slices to send: 1 01:51:41,508 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry_slicer-id-647: Message does not need to be sliced - sending original message 01:51:41,508 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1992989371], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person} with Envelope{sessionId=1, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 01:51:41,508 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1992989371], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person}, enqueuedTicks=855835473320, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 01:51:41,508 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person completed with ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0} 01:51:41,509 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:51:41,509 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:41,509 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:41,509 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry - waiting on 2 shards 01:51:41,509 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onShutDown 01:51:41,509 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onShutDown 01:51:41,509 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:41,509 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Stopping Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:41,509 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:51:41,509 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 01:51:41,509 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 01:51:41,509 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 01:51:41,509 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All shards shutdown - sending PoisonPill to self 01:51:41,509 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 01:51:41,509 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 01:51:41,509 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 01:51:41,510 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 01:51:41,510 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: shutting down 01:51:41,510 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 01:51:41,510 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:41,510 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:41,514 PM [cluster-test-akka.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:41,514 PM [cluster-test-akka.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:41,514 PM [cluster-test-akka.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:41,514 PM [cluster-test-akka.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:41,516 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:51:41,527 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 01:51:41,527 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 01:51:41,527 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-650: Closing 01:51:41,527 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 01:51:41,527 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-651: Closing 01:51:41,527 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 01:51:41,527 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 01:51:41,527 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 01:51:41,528 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:41,528 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:42,527 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [6737073753841734414] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:42,547 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:42,547 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:42,548 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [8816591792776387560] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:42,549 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:42.551] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:42,566 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:42,566 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:42,569 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:42.570] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:42,585 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:42,591 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4663434174921424526] 01:51:42,591 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:51:42,592 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:51:42,592 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:51:42,594 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:42,594 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:42,604 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:51:42,604 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:51:42,604 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:51:42,606 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:51:42,613 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4373320519397212765] 01:51:42,614 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:51:42,615 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:42,615 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:51:42,615 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:42,615 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:42,634 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:42,644 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [6093736658460415925] 01:51:42,644 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:51:42,646 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:42,646 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:51:42,646 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:42,646 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:42,656 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:42,657 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:42,658 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,665 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,666 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#821668195] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:42,670 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:51:42,674 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 01:51:42,674 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:51:42,674 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7d1a86a7 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@54322843 01:51:42,677 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1130aca1 01:51:42,677 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1710165102675, None)) 01:51:42,677 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,677 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,677 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:42,677 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:42,677 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,678 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165102674,0,0)) 01:51:42,678 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@54322843 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@65deae50 01:51:42,678 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 01:51:42,678 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@122613f6 01:51:42,679 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:42,679 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:51:42,679 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:42,679 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,680 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:42,681 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,681 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,681 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546] 01:51:42,681 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 01:51:42,682 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 01:51:42,682 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:42,682 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#19751372 created and ready for shard:member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,682 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#19751372] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:42,682 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-1:snapshot-read#-1956268854] 01:51:42,682 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-1:snapshot-read#-1956268854] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:42,682 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,683 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:42,684 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:42,684 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:42,684 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 01:51:42,684 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:51:42,684 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 01:51:42,684 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 01:51:42,685 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 01:51:42,685 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1678635956] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:42,686 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,686 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 01:51:42,690 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,690 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 01:51:42,690 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:51:42,690 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@18a2b503 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@325ffd82 01:51:42,690 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@775d65ad 01:51:42,691 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1710165102691, None)) 01:51:42,691 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,691 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165102690,0,0)) 01:51:42,691 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,692 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:42,692 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,692 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7b75198d 01:51:42,692 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@325ffd82 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2b0a89df 01:51:42,692 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 01:51:42,692 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:42,692 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,692 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,692 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,692 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1409027204] 01:51:42,692 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 01:51:42,692 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1409027204] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:42,693 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-2:snapshot-read#-2065404667] 01:51:42,693 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#1842822964 created and ready for shard:member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,693 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#1842822964] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:42,693 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-2:snapshot-read#-2065404667] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:42,693 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 01:51:42,693 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:42,693 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:42,693 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 01:51:42,693 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 01:51:42,693 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 01:51:42,693 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:51:42,693 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,693 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 01:51:42,693 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,693 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 01:51:42,742 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#821668195] 01:51:42,742 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: findLocalShard : cars 01:51:42,744 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546] 01:51:42,951 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:42,951 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Received ElectionTimeout - switching to Candidate 01:51:42,951 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 01:51:42,951 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:42,951 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 1 01:51:42,952 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:42,952 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 01:51:42,952 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 01:51:42,952 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:51:42,952 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Follower to Candidate 01:51:42,952 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Follower to Candidate 01:51:43,211 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 01:51:43,211 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,211 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 2 01:51:43,253 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:43,442 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 01:51:43,442 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,442 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 3 01:51:43,612 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:51:43,612 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:51:43,613 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:43,613 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,613 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,613 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:43,613 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,613 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,613 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Peer address for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,613 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,617 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:43,617 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,617 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,617 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:43,617 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,617 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,617 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,659 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:51:43,711 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 01:51:43,711 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,711 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 4 01:51:43,711 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,712 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,713 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:43,713 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Cluster state: [] 01:51:43,713 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:43,713 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:43,713 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=null 01:51:43,713 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:43,713 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,713 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:43,714 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:43,714 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:43,715 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Election: Leader has following peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 01:51:43,715 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:43,715 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:43,715 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:43,715 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 01:51:43,715 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: became leader with frontend state for [] 01:51:43,715 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Candidate to Leader 01:51:43,715 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 01:51:43,715 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:43,715 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:43,715 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Candidate to Leader 01:51:43,715 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 01:51:43,715 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:43,716 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:43,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:51:43,716 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,716 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 01:51:43,716 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:43,716 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:43,716 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 01:51:43,716 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 01:51:43,717 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: -1, nextIndex: 0 01:51:43,717 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:43,751 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:43,753 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:43,756 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,761 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,761 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#786130969] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:43,763 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,764 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,764 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:43,764 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,764 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:43,764 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,764 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:43,764 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,764 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 01:51:43,764 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:51:43,764 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6a689033 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@56f0b991 01:51:43,765 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2920396b 01:51:43,765 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1710165103765, None)) 01:51:43,765 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165103764,0,0)) 01:51:43,765 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@56300972 01:51:43,765 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@56f0b991 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@34eb5db6 01:51:43,765 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 01:51:43,766 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:43,766 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,766 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6e7004e6: creating a new snapshot member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 01:51:43,766 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,766 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,766 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-203170632] 01:51:43,766 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,766 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,766 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-203170632] 01:51:43,766 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} 01:51:43,766 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,766 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:43,766 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,766 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:43,766 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-203170632] 01:51:43,766 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1678635956] 01:51:43,766 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 01:51:43,766 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=0} shard 1 01:51:43,767 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 01:51:43,767 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:43,767 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2063144426], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@34671d09 01:51:43,767 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} scheduling timeout in 2 seconds 01:51:43,767 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 01:51:43,767 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:43,767 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, primaryVersion=13] 01:51:43,767 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-203170632] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:43,767 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-3:snapshot-read#-216879028] 01:51:43,767 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-1039341527 created and ready for shard:member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,767 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-3:snapshot-read#-216879028] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:43,767 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-1039341527] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:43,767 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, primaryVersion=13] 01:51:43,768 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@c84d15e, attempting to connect 01:51:43,769 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} is not yet known 01:51:43,769 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, nanosAgo=14091, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:51:43,770 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:43,770 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:43,770 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:43,770 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@57cb640d instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@17729ae2 01:51:43,770 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:51:43,770 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@17729ae2 01:51:43,770 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 01:51:43,770 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 01:51:43,770 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2063144426], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 01:51:43,770 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:43,770 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2063144426], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1247/0x0000000801301400@6cc9100c 01:51:43,770 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 01:51:43,770 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@101acc5a transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2063144426], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=858097679780} 01:51:43,771 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6e7004e6 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:43,771 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} 01:51:43,771 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@133ce131 01:51:43,771 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4e054e4c superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@6ae87ef0, splicing queue 01:51:43,771 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4e054e4c queue spliced 0 messages 01:51:43,771 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 700.9 μs 01:51:43,771 PM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: allocating new open snapshot member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 01:51:43,771 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-653: slice: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0}} 01:51:43,771 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-653: Message size: 245 bytes, total slices to send: 1 01:51:43,771 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-653: Message does not need to be sliced - sending original message 01:51:43,772 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2063144426], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0}} 01:51:43,772 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2063144426], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=858097679780, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0}} 01:51:43,772 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0} 01:51:43,772 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:51:43,773 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,773 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader)_slicer-id-656: Closing 01:51:43,773 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 01:51:43,773 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,773 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:43,773 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:43,777 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:43,777 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:43,777 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:43,777 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:43,779 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:51:43,780 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:43,780 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:43,780 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 01:51:43,780 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:51:43,780 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,780 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,780 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 01:51:43,780 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 01:51:43,792 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 01:51:43,792 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,792 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 01:51:43,792 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5cc1a1f aborted transaction LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=OPEN} 01:51:43,792 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@3cdd083 01:51:43,792 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:43,792 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@71c47599 transmitting entry ConnectionEntry{request=AbortLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2090313808]}, enqueuedTicks=858123434416} 01:51:43,792 PM [Cleaner-1] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 01:51:43,793 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0} 01:51:43,793 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@111713a9 aborted transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 01:51:43,793 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:51:43,793 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1548096848], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@217b3772 01:51:43,793 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-3-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 01:51:43,793 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} allocated sequence 1 01:51:43,793 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1548096848], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1240/0x0000000801303148@47587893 01:51:43,793 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-3-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-308288568], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:43,793 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@43b1986c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1548096848], modifications=0, protocol=ABORT}, enqueuedTicks=858123700872} 01:51:43,793 PM [Cleaner-1] [DEBUG] MessageSlicer - member-3-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-637: slice: identifier: member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1548096848], modifications=0, protocol=ABORT}} 01:51:43,793 PM [Cleaner-1] [DEBUG] SlicedMessageState - member-3-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-637: Message size: 283 bytes, total slices to send: 1 01:51:43,793 PM [Cleaner-1] [DEBUG] MessageSlicer - member-3-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-637: Message does not need to be sliced - sending original message 01:51:43,793 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0} 01:51:43,793 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3deb5810 aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 01:51:43,793 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:51:43,793 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1992989371], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@7e7a8973 01:51:43,793 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 01:51:43,793 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} allocated sequence 1 01:51:43,793 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1992989371], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1240/0x0000000801303148@515bf57e 01:51:43,793 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1384517031], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:43,793 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7890e7c6 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1992989371], modifications=0, protocol=ABORT}, enqueuedTicks=858124124663} 01:51:43,793 PM [Cleaner-1] [DEBUG] MessageSlicer - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-642: slice: identifier: member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1992989371], modifications=0, protocol=ABORT}} 01:51:43,793 PM [Cleaner-1] [DEBUG] SlicedMessageState - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-642: Message size: 293 bytes, total slices to send: 1 01:51:43,793 PM [Cleaner-1] [DEBUG] MessageSlicer - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-642: Message does not need to be sliced - sending original message 01:51:43,793 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6f5866ec aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 01:51:43,793 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 01:51:43,793 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1992989371], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@7d202d88 01:51:43,793 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 01:51:43,793 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} allocated sequence 1 01:51:43,793 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} sending request ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1992989371], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1240/0x0000000801303148@7a9dfb56 01:51:43,793 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-768104034], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:43,793 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@be832ff transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1992989371], modifications=0, protocol=ABORT}, enqueuedTicks=858124354948} 01:51:43,793 PM [Cleaner-1] [DEBUG] MessageSlicer - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-642: slice: identifier: member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1992989371], modifications=0, protocol=ABORT}} 01:51:43,793 PM [Cleaner-1] [DEBUG] SlicedMessageState - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-642: Message size: 294 bytes, total slices to send: 1 01:51:43,793 PM [Cleaner-1] [DEBUG] MessageSlicer - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-642: Message does not need to be sliced - sending original message 01:51:43,793 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0} 01:51:43,803 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 01:51:43,803 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,804 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6e7004e6: creating a new transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:51:43,804 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:51:43,804 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:51:43,804 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6e7004e6 readied transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 01:51:43,804 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2063144426], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@6441fd4f 01:51:43,804 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:43,804 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@101acc5a transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2063144426], modifications=1, protocol=SIMPLE}, enqueuedTicks=858135106684} 01:51:43,804 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-654: slice: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2063144426], modifications=1, protocol=SIMPLE}} 01:51:43,804 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,804 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 01:51:43,804 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:43,804 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-654: Message size: 408 bytes, total slices to send: 1 01:51:43,804 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-654: Message does not need to be sliced - sending original message 01:51:43,804 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 01:51:43,804 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : true 01:51:43,804 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:51:44,333 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 01:51:44,333 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 01:51:44,333 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 01:51:44,334 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Received ElectionTimeout - switching to Candidate 01:51:44,334 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 01:51:44,334 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:44,334 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 5 01:51:44,334 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:44,334 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:44,334 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:51:44,334 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 01:51:44,334 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 01:51:44,334 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 01:51:44,334 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:51:44,334 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=null, leaderPayloadVersion=-1] 01:51:44,334 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Follower to Candidate 01:51:44,334 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:44,334 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Follower to Candidate 01:51:44,334 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:44,335 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@101acc5a superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@7bdb6e4d, splicing queue 01:51:44,335 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 01:51:44,335 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@101acc5a queue spliced 1 messages 01:51:44,335 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:44,335 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: refreshing backend for shard 1 01:51:44,335 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:44,335 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:44,335 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:44,335 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1678635956] 01:51:44,335 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:44,335 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Candidate, leaderId=null, activeMember=true] 01:51:44,336 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:44,336 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Cluster state: [] 01:51:44,336 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:44,336 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:44,336 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=null 01:51:44,336 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:44,336 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:44,336 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 01:51:44,337 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 01:51:44,337 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 01:51:44,337 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Election: Leader has following peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 01:51:44,337 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:44,338 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:44,338 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 01:51:44,338 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 01:51:44,338 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: became leader with frontend state for [] 01:51:44,338 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 01:51:44,338 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Candidate to Leader 01:51:44,338 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized - notifying 1 OnShardInitialized callbacks 01:51:44,338 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:44,338 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Candidate to Leader 01:51:44,338 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready - notifying 1 OnShardInitialized callbacks 01:51:44,338 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:44,338 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1409027204, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6757bf41] 01:51:44,338 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 01:51:44,338 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1409027204, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6757bf41] 01:51:44,338 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:51:44,338 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:44,338 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 01:51:44,338 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 01:51:44,339 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:44,339 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:44,339 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 01:51:44,339 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@5345fa50, attempting to connect 01:51:44,339 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} is not yet known 01:51:44,339 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, nanosAgo=11410, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:51:44,339 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:51:44,339 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: -1, nextIndex: 0 01:51:44,339 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:44,340 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1409027204], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:51:44,340 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1409027204], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:44,340 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@17729ae2 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2c5607eb 01:51:44,340 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:51:44,340 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} previous state FLUSHED 01:51:44,340 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2c5607eb 01:51:44,340 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 01:51:44,340 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 01:51:44,342 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] LocalProxyTransaction - Not replaying IncrementTransactionSequenceRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2063144426], snapshotOnly=true} 01:51:44,342 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Incrementing sequence org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$IncrementSequence@75dfa8c5 to successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 01:51:44,342 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 replayed 1 successful requests 01:51:44,342 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2c5607eb 01:51:44,342 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 01:51:44,342 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=OPEN} 01:51:44,342 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2063144426], modifications=1, protocol=SIMPLE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=OPEN} 01:51:44,342 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2063144426], modifications=1, protocol=SIMPLE} 01:51:44,342 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SEALED} allocated sequence 0 01:51:44,342 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2063144426], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@6441fd4f 01:51:44,342 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1409027204], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 2 seconds 01:51:44,342 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@34966feb transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2063144426], coordinated=false}, enqueuedTicks=858135106684} 01:51:44,342 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6e7004e6 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1409027204], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:44,342 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: allocating new ready transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 01:51:44,342 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} 01:51:44,342 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} 01:51:44,342 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@b956de6 01:51:44,342 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@30475d1 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@ede2bf5, splicing queue 01:51:44,342 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@30475d1 queue spliced 0 messages 01:51:44,342 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 initiating direct canCommit 01:51:44,342 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Validating transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 01:51:44,342 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 validated 01:51:44,342 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:44,342 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 initiating direct preCommit 01:51:44,342 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Preparing transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 01:51:44,342 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1409027204], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 2.429 ms 01:51:44,342 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 candidate ready 01:51:44,342 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1: canCommit - candidate rootPath: / 01:51:44,342 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1: canCommit - messages: [] 01:51:44,342 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 01:51:44,342 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 prepared 01:51:44,343 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:44,343 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 initiating direct commit 01:51:44,343 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Starting commit for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 01:51:44,343 PM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 192 01:51:44,343 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}] 01:51:44,343 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}] 01:51:44,343 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Replicate message: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:44,343 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:44,343 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:44,343 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}]]] 01:51:44,343 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:44,343 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 submitted to persistence 01:51:44,343 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}] 01:51:44,344 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}]]] 01:51:44,344 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 01:51:44,344 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:44,344 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log Simple{size=192} 01:51:44,344 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}] 01:51:44,344 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 1 01:51:44,344 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=0, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:44,346 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}] 01:51:44,346 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done true 01:51:44,347 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 0, nextIndex: 1 01:51:44,347 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:51:44,347 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Setting last applied to 0 01:51:44,347 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from -1 to 0 01:51:44,347 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 0 data Simple{size=192} 01:51:44,347 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Resuming commit of transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 01:51:44,347 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@73954b2b}, purged=MutableUnsignedLongSet{size=0}} 01:51:44,347 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} 01:51:44,347 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Committed transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 01:51:44,347 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1: commit - successfulFromPrevious: [] 01:51:44,347 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 01:51:44,347 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2063144426], coordinated=false} with Envelope{sessionId=1, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0}} 01:51:44,347 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2063144426], coordinated=false}, enqueuedTicks=858135106684, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0}} 01:51:44,347 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} directCommit completed 01:51:44,347 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR}: initiating purge 01:51:44,347 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 1 01:51:44,347 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2063144426]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@9b0f4f2 01:51:44,347 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#-1196872123] with name member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher 01:51:44,347 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:44,347 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:44,348 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#-1196872123] 01:51:44,348 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2063144426]} 01:51:44,348 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SEALED}: initiating purge 01:51:44,348 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=DONE} allocated sequence 1 01:51:44,348 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2063144426]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@29ea427f 01:51:44,348 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1409027204], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:44,348 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:44,348 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@34966feb transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2063144426]}, enqueuedTicks=858678458246} 01:51:44,348 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: publishChanges: rootPath: / 01:51:44,348 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=0 01:51:44,348 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#-1196872123] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:44,348 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: purging transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 01:51:44,348 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Elapsed time for generation of change events for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher: 95.67 μs 01:51:44,348 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:51:44,348 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, size=84}] 01:51:44,348 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, size=84}] 01:51:44,348 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Replicate message: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:44,349 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Issuing gracefulStop to shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:44,349 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx - waiting on 1 shards 01:51:44,349 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Commit index set to 0 01:51:44,349 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:51:44,349 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Setting last applied to 0 01:51:44,349 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from -1 to 0 01:51:44,349 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 0 data Simple{size=192} 01:51:44,349 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying foreign transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 01:51:44,349 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@7b9fa50f}, purged=MutableUnsignedLongSet{size=0}} 01:51:44,349 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} 01:51:44,349 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Committed transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 01:51:44,350 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#1002998820] with name member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher 01:51:44,350 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#1002998820] 01:51:44,350 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onShutDown 01:51:44,350 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=0 01:51:44,350 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: publishChanges: rootPath: / 01:51:44,350 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#1002998820] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:44,350 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Elapsed time for generation of change events for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher: 75.65 μs 01:51:44,350 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:44,350 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 01:51:44,350 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, size=84}] 01:51:44,350 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 01:51:44,350 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All shards shutdown - sending PoisonPill to self 01:51:44,350 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] LocalActorRef - Message [akka.persistence.JournalProtocol$WriteMessagesSuccessful$] from Actor[akka://cluster-test/system/in-memory-journal#-289534371] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-203170632] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-203170632] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:44,351 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:44,351 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] LocalActorRef - Message [akka.persistence.JournalProtocol$WriteMessageSuccess] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-203170632] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-203170632] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-203170632] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:44,351 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:51:44,351 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:51:44,351 PM [main] [WARN] ActorUtils - ShardManager for testTransactionRetryWithInitialAskTimeoutExOnCreateTx data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:51:44,351 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:51:44,351 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 01:51:44,351 PM [main] [WARN] ActorUtils - ShardManager for testTransactionRetryWithInitialAskTimeoutExOnCreateTx data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:51:44,351 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:51:44,351 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: shutting down 01:51:44,351 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:51:44,352 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 01:51:44,352 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#786130969] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#786130969] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:44,352 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:44,352 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader)_slicer-id-659: Closing 01:51:44,352 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 01:51:44,352 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 01:51:44,353 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:44,353 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:44,788 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4663434174921424526] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:45,365 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:45,365 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:45,367 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:45.368] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:45,369 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4373320519397212765] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:45,374 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:45,374 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:45,376 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:45.377] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:45,400 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:51:45,405 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [5311710227138177008] 01:51:45,405 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:51:45,407 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:51:45,407 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:51:45,416 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:45,416 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:45,416 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:51:45,416 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:51:45,416 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:51:45,422 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:51:45,437 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-906236537244728609] 01:51:45,437 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:51:45,445 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:45,445 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:51:45,449 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:45,449 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:45,462 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:51:45,480 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [5478521852509924965] 01:51:45,480 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:51:45,485 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:45,485 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:51:45,485 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:45,485 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:45,493 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:45,494 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:45,496 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithMultipleShards 01:51:45,503 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#1020575016] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:45,504 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 01:51:45,505 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: no initial generation override, starting from 0 01:51:45,505 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithMultipleShards 01:51:45,505 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:51:45,505 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:45,505 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@43e7852b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4fabe608 01:51:45,505 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@aa96199 01:51:45,505 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:51:45,505 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, 0, 1710165105505, None)) 01:51:45,506 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:45,506 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:45,506 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:51:45,506 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165105504,0,0)) 01:51:45,506 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4fabe608 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2b62878a 01:51:45,506 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} started 01:51:45,506 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:45,506 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6c4a00bd 01:51:45,507 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:45,507 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:45,507 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:45,509 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithMultipleShards 01:51:45,512 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:51:45,514 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:45,517 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:45,517 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:45,517 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards on shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#957377664] 01:51:45,517 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:51:45,518 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-1-shard-cars-testReadWriteTransactionWithMultipleShards, persistent : true 01:51:45,518 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadWriteTransactionWithMultipleShards with journal batch size 1 01:51:45,518 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-1-shard-people-testReadWriteTransactionWithMultipleShards, persistent : true 01:51:45,519 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-1-shard-people-testReadWriteTransactionWithMultipleShards with journal batch size 1 01:51:45,519 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#957377664] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:45,519 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1228987935] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:45,519 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#-1330135196 created and ready for shard:member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:45,519 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#-1330135196] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:45,520 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:51:45,520 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:51:45,520 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards on shard member-1-shard-people-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1228987935] 01:51:45,528 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-1460689989] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:45,529 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 01:51:45,531 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#2036613801] 01:51:45,531 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#2036613801] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:45,539 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:51:45,541 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:45,546 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-1121305949 created and ready for shard:member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:51:45,546 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-1121305949] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:45,547 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:45,547 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:45,547 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:45,547 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:45,547 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [cars] 01:51:45,547 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is initialized 01:51:45,547 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [people] 01:51:45,547 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is initialized 01:51:45,548 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:45,548 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:45,548 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 01:51:45,548 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 01:51:45,548 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 01:51:45,548 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 01:51:45,548 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 01:51:45,548 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 01:51:45,548 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from null to Follower 01:51:45,548 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from null to Follower 01:51:45,548 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#1904924862] 01:51:45,548 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#1904924862] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:45,553 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithMultipleShards 01:51:45,553 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:45,554 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:51:45,554 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:45,554 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:45,554 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:51:45,554 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: no initial generation override, starting from 0 01:51:45,554 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:51:45,554 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6d0e11df to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7cdb5edd 01:51:45,554 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@78e4d59e 01:51:45,554 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, 0, 1710165105554, None)) 01:51:45,555 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165105553,0,0)) 01:51:45,555 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7606cc6c 01:51:45,555 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7cdb5edd to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@24c79ef0 01:51:45,555 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} started 01:51:45,555 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:45,555 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:45,555 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:45,555 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:45,556 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards on shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards#-841961024] 01:51:45,556 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:51:45,556 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards#-841961024] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:45,556 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testReadWriteTransactionWithMultipleShards, persistent : true 01:51:45,557 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadWriteTransactionWithMultipleShards with journal batch size 1 01:51:45,557 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-people-testReadWriteTransactionWithMultipleShards, persistent : true 01:51:45,557 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-people-testReadWriteTransactionWithMultipleShards with journal batch size 1 01:51:45,558 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#-183404196] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:45,559 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:51:45,559 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:51:45,559 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards on shard member-2-shard-people-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards#-1377388262] 01:51:45,560 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#-183404196 created and ready for shard:member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:45,560 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-466760180 created and ready for shard:member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:51:45,560 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#959571625] 01:51:45,561 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards#-1377388262] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:45,561 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-466760180] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:45,561 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#959571625] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:45,561 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-866153737] 01:51:45,561 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-866153737] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:45,563 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:45,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:45,563 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:45,563 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:45,563 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [cars] 01:51:45,563 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards is initialized 01:51:45,563 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:45,563 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 01:51:45,563 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [people] 01:51:45,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:45,563 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards is initialized 01:51:45,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 01:51:45,563 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:45,563 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 01:51:45,563 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:51:45,563 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 01:51:45,563 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 01:51:45,563 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 01:51:45,563 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-2-shard-cars-testReadWriteTransactionWithMultipleShards from null to Follower 01:51:45,563 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-2-shard-people-testReadWriteTransactionWithMultipleShards from null to Follower 01:51:45,605 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#1020575016] 01:51:45,606 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: findLocalShard : cars 01:51:45,606 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#957377664] 01:51:45,787 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:45,787 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:51:45,787 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadWriteTransactionWithMultipleShards] 01:51:45,787 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:45,787 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 01:51:45,787 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:45,787 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Closing 01:51:45,787 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:45,787 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 01:51:45,788 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from Follower to Candidate 01:51:45,788 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from Follower to Candidate 01:51:45,857 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:45,857 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:51:45,857 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testReadWriteTransactionWithMultipleShards] 01:51:45,857 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:51:45,857 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 01:51:45,857 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:45,857 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Closing 01:51:45,857 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:51:45,857 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 01:51:45,857 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from Follower to Candidate 01:51:45,857 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from Follower to Candidate 01:51:46,16 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:51:46,16 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:46,16 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 01:51:46,97 PM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:46,126 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:51:46,126 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:51:46,126 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 01:51:46,167 PM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:46,337 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:51:46,337 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:46,337 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 01:51:46,357 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:51:46,357 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:51:46,357 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 01:51:46,428 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:51:46,428 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:51:46,429 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:46,429 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:46,429 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:46,429 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards 01:51:46,429 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:51:46,429 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:46,429 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:46,429 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:46,429 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:51:46,429 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:51:46,429 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:46,429 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:51:46,434 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:46,434 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:46,434 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:46,434 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards 01:51:46,434 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:51:46,434 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:46,434 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:46,434 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:46,434 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:51:46,434 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:51:46,495 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:51:46,647 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:51:46,647 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:51:46,647 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 01:51:46,647 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:51:46,649 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:46,649 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Cluster state: [] 01:51:46,649 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:46,649 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:46,649 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 01:51:46,649 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:46,649 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:51:46,649 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:46,651 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:46,651 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:46,651 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testReadWriteTransactionWithMultipleShards] 01:51:46,651 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:46,651 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:46,651 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:51:46,651 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: became leader with frontend state for [] 01:51:46,651 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:51:46,651 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:46,651 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:46,651 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from Candidate to Leader 01:51:46,651 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from Candidate to Leader 01:51:46,652 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Closing 01:51:46,652 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testReadWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:51:46,652 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:51:46,653 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:46,653 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:46,653 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testReadWriteTransactionWithMultipleShards status sync done false 01:51:46,654 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:51:46,654 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:46,657 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:51:46,657 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:46,657 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 01:51:46,657 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:46,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:46,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Cluster state: [] 01:51:46,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:46,658 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:46,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 01:51:46,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:46,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:46,658 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:46,659 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:46,659 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:46,659 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadWriteTransactionWithMultipleShards] 01:51:46,659 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:46,659 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:46,659 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:51:46,659 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: became leader with frontend state for [] 01:51:46,659 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from Candidate to Leader 01:51:46,659 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:51:46,659 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:46,659 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:46,659 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from Candidate to Leader 01:51:46,659 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All Shards are ready - data store testReadWriteTransactionWithMultipleShards is ready 01:51:46,660 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Closing 01:51:46,660 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testReadWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:46,660 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithMultipleShards status sync done false 01:51:46,660 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:51:46,660 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:46,660 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:46,660 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All Shards are ready - data store testReadWriteTransactionWithMultipleShards is ready 01:51:46,660 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:51:46,660 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:46,721 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#1020575016] 01:51:46,721 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: findLocalShard : people 01:51:46,721 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1228987935] 01:51:46,721 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5e14e23: creating a new transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:46,721 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} 01:51:46,721 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:46,721 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:46,721 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-1460689989] 01:51:46,721 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:46,721 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} shard 1 01:51:46,722 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:46,722 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:46,722 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 01:51:46,722 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} 01:51:46,722 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:51:46,722 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:51:46,722 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 01:51:46,722 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-1460689989] 01:51:46,722 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} shard 2 01:51:46,722 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:46,722 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:51:46,722 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5e14e23 readied transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:51:46,722 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@26a3a635 01:51:46,722 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:51:46,722 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:51:46,722 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6ecab2c7, attempting to connect 01:51:46,722 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@2d4dad82 01:51:46,722 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} scheduling timeout in 30 seconds 01:51:46,722 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 01:51:46,722 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 01:51:46,722 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@7f085a44, attempting to connect 01:51:46,723 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} is not yet known 01:51:46,723 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, nanosAgo=12431, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:51:46,724 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} is not yet known 01:51:46,724 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, nanosAgo=7530, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:51:46,726 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:46,726 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#957377664], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:46,726 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#957377664], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:46,726 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@f8cd5d7 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3f89f63c 01:51:46,726 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:51:46,726 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3f89f63c 01:51:46,726 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:46,726 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} 01:51:46,726 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} 01:51:46,726 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:46,726 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2239/0x00000008016abcb8@43291649 01:51:46,726 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#957377664], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:46,726 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4bbbe15b transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=861052891057} 01:51:46,726 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-663: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], modifications=1, protocol=THREE_PHASE}} 01:51:46,727 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-663: Message size: 396 bytes, total slices to send: 1 01:51:46,727 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-663: Message does not need to be sliced - sending original message 01:51:46,727 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5e14e23 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#957377664], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:46,727 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 01:51:46,727 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@2d5882fd 01:51:46,727 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@26a5b573 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3d846af5, splicing queue 01:51:46,727 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@26a5b573 queue spliced 0 messages 01:51:46,727 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#957377664], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.088 ms 01:51:46,728 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:46,728 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1228987935], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:51:46,728 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1228987935], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:46,728 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1d618493 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1972fd95 01:51:46,728 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} previous state FLUSHED 01:51:46,728 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1972fd95 01:51:46,728 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:46,728 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} 01:51:46,728 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} 01:51:46,728 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:51:46,728 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2239/0x00000008016abcb8@cbace9c 01:51:46,728 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1228987935], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:51:46,728 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5a08ad05 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=861052971909} 01:51:46,728 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-663: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], modifications=1, protocol=THREE_PHASE}} 01:51:46,728 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-663: Message size: 401 bytes, total slices to send: 1 01:51:46,728 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-663: Message does not need to be sliced - sending original message 01:51:46,728 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5e14e23 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1228987935], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:46,728 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 01:51:46,728 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@573017ea 01:51:46,728 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@56d5d557 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@5d2a3101, splicing queue 01:51:46,728 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@56d5d557 queue spliced 0 messages 01:51:46,729 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1228987935], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 790.4 μs 01:51:46,729 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: allocating new open transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:46,729 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:46,729 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 ready 01:51:46,729 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: transitioned member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 to ready 01:51:46,729 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating canCommit 01:51:46,730 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:46,730 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 validated 01:51:46,730 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:46,730 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed canCommit 01:51:46,730 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testReadWriteTransactionWithMultipleShards: allocating new open transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:46,731 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:46,731 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 ready 01:51:46,731 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: transitioned member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 to ready 01:51:46,731 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating canCommit 01:51:46,731 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:46,731 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 validated 01:51:46,731 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:46,731 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed canCommit 01:51:46,731 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0}} 01:51:46,731 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=861052891057, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0}} 01:51:46,731 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 01:51:46,731 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0}} 01:51:46,731 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=861052971909, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0}} 01:51:46,731 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} canCommit completed 01:51:46,731 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:51:46,731 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@1e55f842 01:51:46,731 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} 01:51:46,731 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} allocated sequence 1 01:51:46,731 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x0000000801303580@25fc67b3 01:51:46,731 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#957377664], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:46,731 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4bbbe15b transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]}, enqueuedTicks=861062373481} 01:51:46,731 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 01:51:46,731 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@651dff33 01:51:46,731 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} 01:51:46,731 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} allocated sequence 1 01:51:46,731 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x0000000801303580@167a36f5 01:51:46,731 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1228987935], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:46,731 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5a08ad05 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]}, enqueuedTicks=861062508424} 01:51:46,732 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating preCommit 01:51:46,732 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:46,732 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 candidate ready 01:51:46,732 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1: canCommit - candidate rootPath: / 01:51:46,732 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating preCommit 01:51:46,732 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1: canCommit - messages: [] 01:51:46,732 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 01:51:46,732 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:46,732 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 prepared 01:51:46,732 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:46,732 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed preCommit 01:51:46,732 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 candidate ready 01:51:46,732 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2: canCommit - candidate rootPath: / 01:51:46,732 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2: canCommit - messages: [] 01:51:46,732 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2: preCommit - successfulFromPrevious: [] 01:51:46,732 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 prepared 01:51:46,732 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:46,732 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed preCommit 01:51:46,733 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1}} 01:51:46,733 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]}, enqueuedTicks=861062373481, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1}} 01:51:46,733 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:51:46,733 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1}} 01:51:46,733 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]}, enqueuedTicks=861062508424, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1}} 01:51:46,733 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:51:46,739 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:51:46,739 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@615b6137 01:51:46,739 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} 01:51:46,740 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} allocated sequence 2 01:51:46,740 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@615b6137 01:51:46,740 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#957377664], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:46,740 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4bbbe15b transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]}, enqueuedTicks=861070572993} 01:51:46,740 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 2 01:51:46,740 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@6bb5fe48 01:51:46,740 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} 01:51:46,740 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} allocated sequence 2 01:51:46,740 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@6bb5fe48 01:51:46,740 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1228987935], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:46,740 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5a08ad05 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]}, enqueuedTicks=861070673136} 01:51:46,741 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating commit 01:51:46,741 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:46,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating commit 01:51:46,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:46,741 PM [cluster-test-shard-dispatcher-26] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 180 01:51:46,741 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:51:46,741 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:51:46,741 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:46,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 184 01:51:46,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 01:51:46,741 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:46,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 01:51:46,741 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:46,741 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 01:51:46,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:46,741 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 submitted to persistence 01:51:46,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:46,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:51:46,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}]]] 01:51:46,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 submitted to persistence 01:51:46,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:51:46,741 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 01:51:46,742 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 01:51:46,742 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}]]] 01:51:46,742 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:51:46,742 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:46,742 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:51:46,742 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=180} 01:51:46,742 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:46,742 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:51:46,742 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=184} 01:51:46,742 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 01:51:46,742 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 1 01:51:46,742 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 1 01:51:46,742 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:46,742 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:46,743 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:51:46,743 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 01:51:46,743 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithMultipleShards status sync done true 01:51:46,743 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testReadWriteTransactionWithMultipleShards status sync done true 01:51:46,743 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:51:46,743 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:51:46,743 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 01:51:46,743 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:51:46,743 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:51:46,743 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=180} 01:51:46,743 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:51:46,743 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:46,743 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 01:51:46,743 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:51:46,743 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=184} 01:51:46,743 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:46,743 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@d1b3f7b}, purged=MutableUnsignedLongSet{size=0}} 01:51:46,743 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 01:51:46,743 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:46,743 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1: commit - successfulFromPrevious: [] 01:51:46,743 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@30f86a04}, purged=MutableUnsignedLongSet{size=0}} 01:51:46,743 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:51:46,743 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 01:51:46,743 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:46,744 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2: commit - successfulFromPrevious: [] 01:51:46,744 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 01:51:46,744 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-529262673] with name member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher 01:51:46,744 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#965273723] with name member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher 01:51:46,744 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:46,744 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:46,744 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:51:46,745 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-529262673] 01:51:46,745 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:51:46,745 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-529262673] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:46,745 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 81.84 μs 01:51:46,745 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#965273723] 01:51:46,745 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:51:46,745 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#965273723] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:46,745 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 83.34 μs 01:51:46,746 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:51:46,746 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 0 01:51:46,746 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 0 01:51:46,746 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:51:46,746 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 01:51:46,746 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:51:46,746 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:51:46,746 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 01:51:46,746 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=180} 01:51:46,746 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:51:46,746 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=184} 01:51:46,746 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:46,746 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:46,747 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@36bcbd1c}, purged=MutableUnsignedLongSet{size=0}} 01:51:46,747 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 01:51:46,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@72bb2e0d}, purged=MutableUnsignedLongSet{size=0}} 01:51:46,747 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:46,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 01:51:46,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:46,747 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1235501092] with name member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher 01:51:46,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-416020582] with name member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher 01:51:46,747 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:51:46,747 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-416020582] 01:51:46,747 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:51:46,747 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 55.55 μs 01:51:46,747 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-416020582] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:46,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:51:46,747 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1235501092] 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 46.84 μs 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1235501092] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2}} 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]}, enqueuedTicks=861070572993, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2}} 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@f8cd5d7 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@5abbf7fc 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED}: initiating purge 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} allocated sequence 3 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@494ef321 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#957377664], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4bbbe15b transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]}, enqueuedTicks=861078879598} 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2}} 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]}, enqueuedTicks=861070673136, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2}} 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} doCommit completed 01:51:46,748 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5e14e23: creating a new snapshot member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1d618493 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR}: initiating purge 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 3 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@18dc976f 01:51:46,748 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:51:46,748 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} 01:51:46,748 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@17118bcc 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED}: initiating purge 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} allocated sequence 3 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@1ddd979f 01:51:46,748 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4bbbe15b transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=861079468393} 01:51:46,748 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1228987935], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:46,749 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5a08ad05 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]}, enqueuedTicks=861079374411} 01:51:46,750 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: purging transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:46,750 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: purging transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:46,750 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 01:51:46,750 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 01:51:46,750 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 01:51:46,750 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:46,750 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 01:51:46,750 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:46,750 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: allocating new open snapshot member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1 01:51:46,750 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithMultipleShards_slicer-id-661: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 01:51:46,750 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 01:51:46,750 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 01:51:46,750 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SlicedMessageState - member-1-shard-cars-testReadWriteTransactionWithMultipleShards_slicer-id-661: Message size: 337 bytes, total slices to send: 1 01:51:46,750 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithMultipleShards_slicer-id-661: Message does not need to be sliced - sending original message 01:51:46,751 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 01:51:46,751 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=861079468393, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 01:51:46,751 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0} 01:51:46,751 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:51:46,751 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} allocated sequence 0 01:51:46,751 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@38ddebea 01:51:46,751 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5a08ad05 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people}, enqueuedTicks=861081782010} 01:51:46,751 PM [cluster-test-shard-dispatcher-26] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testReadWriteTransactionWithMultipleShards: allocating new open snapshot member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2 01:51:46,751 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-people-testReadWriteTransactionWithMultipleShards_slicer-id-662: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, message: Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 01:51:46,751 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SlicedMessageState - member-1-shard-people-testReadWriteTransactionWithMultipleShards_slicer-id-662: Message size: 341 bytes, total slices to send: 1 01:51:46,751 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-people-testReadWriteTransactionWithMultipleShards_slicer-id-662: Message does not need to be sliced - sending original message 01:51:46,752 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people} with Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 01:51:46,752 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people}, enqueuedTicks=861081782010, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 01:51:46,752 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people completed with ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0} 01:51:46,752 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:51:46,752 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:46,752 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:51:46,753 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithMultipleShards - waiting on 2 shards 01:51:46,753 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onShutDown 01:51:46,753 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onShutDown 01:51:46,753 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Initiating leader transfer 01:51:46,753 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Initiating leader transfer 01:51:46,753 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@46553af8 01:51:46,753 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3fdfa8a0 01:51:46,753 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@46553af8 01:51:46,753 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 01:51:46,753 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:51:46,753 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3fdfa8a0 01:51:46,753 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:51:46,753 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 01:51:46,753 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}]]] 01:51:46,753 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:51:46,753 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:46,753 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:51:46,753 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}]]] 01:51:46,753 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:46,753 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:51:46,753 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:51:46,753 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:46,753 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:46,754 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards] 01:51:46,754 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:51:46,754 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:51:46,754 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards] 01:51:46,754 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}]]] 01:51:46,754 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:51:46,754 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:51:46,754 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72} 01:51:46,754 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 01:51:46,754 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 2 01:51:46,754 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:46,754 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}]]] 01:51:46,754 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:51:46,754 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:51:46,754 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72} 01:51:46,754 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 01:51:46,754 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 2 01:51:46,754 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:46,754 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 01:51:46,754 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5a08ad05 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@48f03c9a, splicing queue 01:51:46,754 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1228987935], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:51:46,754 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5a08ad05 queue spliced 1 messages 01:51:46,754 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1228987935], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1228987935], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:46,754 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: refreshing backend for shard 2 01:51:46,754 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1228987935], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:51:46,754 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:51:46,754 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:51:46,754 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-1460689989] 01:51:46,754 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:51:46,755 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 01:51:46,755 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:51:46,755 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:51:46,755 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 01:51:46,755 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:46,755 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:46,755 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4bbbe15b superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@79c367bd, splicing queue 01:51:46,755 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#957377664], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:46,755 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4bbbe15b queue spliced 1 messages 01:51:46,755 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#957377664], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#957377664], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:46,755 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: refreshing backend for shard 1 01:51:46,755 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#957377664], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:46,755 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:46,755 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:46,755 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-1460689989] 01:51:46,755 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:51:46,755 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:51:46,755 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 01:51:46,755 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:51:46,755 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72} 01:51:46,755 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: replication of member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@2faf5f57 01:51:46,755 PM [cluster-test-shard-dispatcher-19] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testReadWriteTransactionWithMultipleShards: finished purging transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:46,755 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 01:51:46,755 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:46,755 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:46,755 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:51:46,755 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:51:46,755 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:46,755 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:51:46,755 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testReadWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:51:46,755 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:51:46,755 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:51:46,755 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:46,755 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 01:51:46,755 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:51:46,755 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72} 01:51:46,755 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Leader transfer complete 01:51:46,755 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: replication of member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@6b4c83fd 01:51:46,755 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:51:46,755 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: finished purging transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:46,755 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 01:51:46,755 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:46,755 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:51:46,756 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:46,756 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:51:46,756 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:51:46,756 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:46,756 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Leader transfer complete 01:51:46,756 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:51:46,756 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} not found in inflight queue, checking pending queue 01:51:46,756 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} 01:51:46,756 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]}, enqueuedTicks=861079374411, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} 01:51:46,756 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE}: purge completed 01:51:46,756 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1972fd95 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} 01:51:46,756 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR}: purge completed 01:51:46,756 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1d618493 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 01:51:46,757 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} not found in inflight queue, checking pending queue 01:51:46,757 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} 01:51:46,757 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1472770649]}, enqueuedTicks=861078879598, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} 01:51:46,757 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE}: purge completed 01:51:46,757 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3f89f63c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} 01:51:46,757 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:51:46,757 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@f8cd5d7 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 01:51:46,757 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 1 01:51:46,757 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:51:46,757 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 1 01:51:46,757 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 01:51:46,757 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:51:46,757 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:51:46,757 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72} 01:51:46,757 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 01:51:46,757 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:51:46,757 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72} 01:51:46,757 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 01:51:46,757 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 01:51:46,757 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:51:46,757 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:51:46,757 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:51:46,757 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:51:46,757 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:51:46,757 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:51:47,167 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 01:51:47,167 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#957377664]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:51:47,167 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:47,167 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader)_slicer-id-667: Closing 01:51:47,168 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 01:51:47,168 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader)_slicer-id-666: Closing 01:51:47,168 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:51:47,168 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:51:47,168 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 01:51:47,168 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 01:51:47,168 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 01:51:47,168 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: shutting down 01:51:47,168 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 01:51:47,168 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 01:51:47,169 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#1020575016] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#1020575016] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:47,169 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:47,169 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:47,174 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:47,174 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:47,175 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:47,175 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:47,177 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:51:47,188 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 01:51:47,188 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 01:51:47,188 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Closing 01:51:47,188 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 01:51:47,188 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Closing 01:51:47,188 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:51:47,188 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:51:47,189 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 01:51:47,189 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:47,189 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:48,185 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [5311710227138177008] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:48,205 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-906236537244728609] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:48,209 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:48,209 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:48,210 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:48.212] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:48,236 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:48,236 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:48,239 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:48.240] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:48,263 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:48,268 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [6674873829856076223] 01:51:48,269 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:51:48,270 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:51:48,270 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:51:48,270 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:48,270 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:48,280 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:51:48,280 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:51:48,280 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:51:48,281 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:51:48,290 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [3933556073897378099] 01:51:48,290 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:51:48,302 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:48,302 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:51:48,302 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:48,302 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:48,310 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:48,326 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [2365291670113967049] 01:51:48,326 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:51:48,330 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:48,330 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:51:48,330 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:48,330 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:48,337 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:48,338 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:48,340 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLargeReadReplySlicing 01:51:48,347 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-1052584595] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:48,347 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:51:48,350 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:48,359 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:51:48,360 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:48,366 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-testLargeReadReplySlicing 01:51:48,368 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: no initial generation override, starting from 0 01:51:48,368 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testLargeReadReplySlicing, generation=0} 01:51:48,368 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLargeReadReplySlicing: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@35368892 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@57b30b37 01:51:48,368 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4eefe0 01:51:48,368 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testLargeReadReplySlicing, 0, 1710165108368, None)) 01:51:48,368 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165108367,0,0)) 01:51:48,369 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLargeReadReplySlicing: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@57b30b37 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@379cf478 01:51:48,369 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testLargeReadReplySlicing, generation=0} started 01:51:48,369 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:48,371 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:48,373 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLargeReadReplySlicing 01:51:48,374 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-testLargeReadReplySlicing 01:51:48,374 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Creating local shard: member-1-shard-cars-testLargeReadReplySlicing 01:51:48,374 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:48,374 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 01:51:48,374 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@20b7c2d9 01:51:48,374 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:48,374 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testLargeReadReplySlicing 01:51:48,374 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 01:51:48,374 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 01:51:48,374 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: updated peer member-1-shard-cars-testLargeReadReplySlicing on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing on shard member-1-shard-cars-testLargeReadReplySlicing whose actor address is Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123] 01:51:48,375 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-1-shard-cars-testLargeReadReplySlicing, persistent : true 01:51:48,375 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-1-shard-cars-testLargeReadReplySlicing with journal batch size 1 01:51:48,376 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing-notifier#-832821208 created and ready for shard:member-1-shard-cars-testLargeReadReplySlicing 01:51:48,376 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing/shard-member-1:snapshot-read#-2065654622] 01:51:48,376 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:48,376 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1477913742] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:48,377 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing-notifier#-832821208] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:48,377 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing/shard-member-1:snapshot-read#-2065654622] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:48,377 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:48,377 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testLargeReadReplySlicing 01:51:48,377 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:48,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Initializing shard [cars] 01:51:48,377 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLargeReadReplySlicing: onLeadershipChange, isLeader: false, hasLeader : false 01:51:48,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-testLargeReadReplySlicing is initialized 01:51:48,377 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLargeReadReplySlicing is no longer the leader 01:51:48,377 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLargeReadReplySlicing , received role change from null to Follower 01:51:48,377 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLargeReadReplySlicing , registered listener akka://cluster-test/user/shardmanager-testLargeReadReplySlicing 01:51:48,377 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received role changed for member-1-shard-cars-testLargeReadReplySlicing from null to Follower 01:51:48,377 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testLargeReadReplySlicing 01:51:48,377 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: no initial generation override, starting from 0 01:51:48,377 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} 01:51:48,377 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Creating local shard: member-2-shard-cars-testLargeReadReplySlicing 01:51:48,377 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLargeReadReplySlicing: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@125516ec to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@232fbc9d 01:51:48,378 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@65167291 01:51:48,378 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:48,378 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 01:51:48,378 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testLargeReadReplySlicing, 0, 1710165108378, None)) 01:51:48,378 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165108377,0,0)) 01:51:48,378 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7fab5a83 01:51:48,378 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLargeReadReplySlicing: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@232fbc9d to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@f53be96 01:51:48,378 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} started 01:51:48,379 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:48,379 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testLargeReadReplySlicing 01:51:48,379 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 01:51:48,379 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 01:51:48,379 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: updated peer member-1-shard-cars-testLargeReadReplySlicing on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing on shard member-2-shard-cars-testLargeReadReplySlicing whose actor address is Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing#334986012] 01:51:48,379 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testLargeReadReplySlicing, persistent : true 01:51:48,379 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing#334986012] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:48,380 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing-notifier#498137953 created and ready for shard:member-2-shard-cars-testLargeReadReplySlicing 01:51:48,380 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing-notifier#498137953] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:48,380 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing/shard-member-2:snapshot-read#1715331414] 01:51:48,380 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing/shard-member-2:snapshot-read#1715331414] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:48,380 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testLargeReadReplySlicing with journal batch size 1 01:51:48,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:48,380 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testLargeReadReplySlicing: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:48,380 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Initializing shard [cars] 01:51:48,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLargeReadReplySlicing: onLeadershipChange, isLeader: false, hasLeader : false 01:51:48,380 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLargeReadReplySlicing is initialized 01:51:48,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLargeReadReplySlicing is no longer the leader 01:51:48,381 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLargeReadReplySlicing , received role change from null to Follower 01:51:48,381 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLargeReadReplySlicing set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 01:51:48,381 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLargeReadReplySlicing , registered listener akka://cluster-test/user/shardmanager-testLargeReadReplySlicing 01:51:48,381 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received role changed for member-2-shard-cars-testLargeReadReplySlicing from null to Follower 01:51:48,429 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-1052584595] 01:51:48,429 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: findLocalShard : cars 01:51:48,429 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123] 01:51:48,639 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:48,639 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Follower): Received ElectionTimeout - switching to Candidate 01:51:48,639 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testLargeReadReplySlicing] 01:51:48,639 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Set currentTerm=1, votedFor=member-1-shard-cars-testLargeReadReplySlicing 01:51:48,639 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Starting new election term 1 01:51:48,639 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:48,639 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing (Follower): Closing 01:51:48,639 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLargeReadReplySlicing: onLeadershipChange, isLeader: false, hasLeader : false 01:51:48,639 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLargeReadReplySlicing is no longer the leader 01:51:48,639 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLargeReadReplySlicing , received role change from Follower to Candidate 01:51:48,639 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received role changed for member-1-shard-cars-testLargeReadReplySlicing from Follower to Candidate 01:51:48,869 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Received ElectionTimeout 01:51:48,869 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Set currentTerm=2, votedFor=member-1-shard-cars-testLargeReadReplySlicing 01:51:48,869 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Starting new election term 2 01:51:48,931 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:49,169 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Received ElectionTimeout 01:51:49,169 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Set currentTerm=3, votedFor=member-1-shard-cars-testLargeReadReplySlicing 01:51:49,169 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Starting new election term 3 01:51:49,290 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:51:49,290 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:51:49,291 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:49,291 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLargeReadReplySlicing/member-3-shard-cars-testLargeReadReplySlicing 01:51:49,291 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLargeReadReplySlicing/member-3-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 01:51:49,291 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:49,291 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 01:51:49,291 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 01:51:49,291 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLargeReadReplySlicing set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 01:51:49,300 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:49,300 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLargeReadReplySlicing/member-3-shard-cars-testLargeReadReplySlicing 01:51:49,300 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLargeReadReplySlicing/member-3-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 01:51:49,300 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:49,300 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 01:51:49,300 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 01:51:49,344 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:51:49,458 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Received ElectionTimeout 01:51:49,458 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Set currentTerm=4, votedFor=member-1-shard-cars-testLargeReadReplySlicing 01:51:49,459 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Starting new election term 4 01:51:49,459 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLargeReadReplySlicing, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testLargeReadReplySlicing 01:51:49,460 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:49,460 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): Cluster state: [] 01:51:49,460 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testLargeReadReplySlicing, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:49,460 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testLargeReadReplySlicing, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:49,460 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: Set currentTerm=4, votedFor=null 01:51:49,460 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testLargeReadReplySlicing, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:49,460 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: Set currentTerm=4, votedFor=member-1-shard-cars-testLargeReadReplySlicing 01:51:49,460 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:49,466 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:49,466 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:49,466 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Election: Leader has following peers: [member-2-shard-cars-testLargeReadReplySlicing] 01:51:49,466 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLargeReadReplySlicing: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:49,466 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:49,466 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLargeReadReplySlicing: onLeadershipChange, isLeader: true, hasLeader : true 01:51:49,466 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: became leader with frontend state for [] 01:51:49,467 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testLargeReadReplySlicing: Closing 01:51:49,467 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: getLeaderAddress leaderId = member-1-shard-cars-testLargeReadReplySlicing peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 01:51:49,467 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLargeReadReplySlicing, leaderId=member-1-shard-cars-testLargeReadReplySlicing, leaderPayloadVersion=13] 01:51:49,467 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:49,467 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:49,467 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: All Shards are ready - data store testLargeReadReplySlicing is ready 01:51:49,467 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing Received follower initial sync status for member-2-shard-cars-testLargeReadReplySlicing status sync done false 01:51:49,468 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLargeReadReplySlicing updated: matchIndex: -1, nextIndex: 0 01:51:49,468 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLargeReadReplySlicing: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:49,468 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLargeReadReplySlicing , received role change from Candidate to Leader 01:51:49,468 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLargeReadReplySlicing, leaderId=member-1-shard-cars-testLargeReadReplySlicing, leaderPayloadVersion=13] 01:51:49,468 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:49,468 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:49,468 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received role changed for member-1-shard-cars-testLargeReadReplySlicing from Candidate to Leader 01:51:49,468 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: All Shards are ready - data store testLargeReadReplySlicing is ready 01:51:49,485 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@43976b57: creating a new transaction member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-0 01:51:49,485 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} 01:51:49,485 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:49,485 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:49,485 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1477913742] 01:51:49,485 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, history=0, cookie=0} shard 1 01:51:49,486 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1 for transaction member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-0 01:51:49,486 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:49,486 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} allocated sequence 1 01:51:49,486 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1775345402], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@7fb0c397 01:51:49,486 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:51:49,486 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1775345402], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@6c460333 01:51:49,486 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:49,486 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing, primaryVersion=13] 01:51:49,486 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing, primaryVersion=13] 01:51:49,486 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@67ba4f3c, attempting to connect 01:51:49,487 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: client ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} is not yet known 01:51:49,487 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, nanosAgo=12980, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} 01:51:49,488 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:49,489 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:49,489 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:49,489 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5ccfc0db instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@665eb1b2 01:51:49,489 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:51:49,489 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@665eb1b2 01:51:49,489 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1 for transaction member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1 01:51:49,489 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} 01:51:49,489 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1775345402], modifications=1} to successor RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} 01:51:49,489 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:49,489 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1775345402], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} 01:51:49,489 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} enqueing request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1775345402], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2271/0x00000008016c4688@1ecd5c23 01:51:49,489 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:49,489 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@53d03aa1 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1775345402], modifications=1}, enqueuedTicks=863816824084} 01:51:49,489 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-670: slice: identifier: member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1775345402], modifications=1}} 01:51:49,490 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-670: Message size: 449 bytes, total slices to send: 5 01:51:49,490 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-670: getNextSlice: total size: 449, offset: 0, size: 100, index: 1 01:51:49,496 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-670: Sending first slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77], data.length=100, sliceIndex=1, totalSlices=5, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test/user/$a#1775345402]] 01:51:49,496 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} allocated sequence 1 01:51:49,497 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1775345402], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1247/0x0000000801301400@5b5910eb 01:51:49,497 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@53d03aa1 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1775345402], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=863816824084} 01:51:49,497 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@53d03aa1 cannot transmit request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1775345402], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} - delaying it 01:51:49,497 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@43976b57 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:49,497 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} 01:51:49,497 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@399c4d14 01:51:49,497 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4120f4aa superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1fd280ee, splicing queue 01:51:49,497 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4120f4aa queue spliced 0 messages 01:51:49,497 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 8.100 ms 01:51:49,504 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77], data.length=100, sliceIndex=1, totalSlices=5, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1775345402]] 01:51:49,505 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Received first slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77] - creating AssembledMessageState 01:51:49,507 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77], sliceIndex: 1, lastSliceIndex: 0, assembledSize: 0, sliceHashCode: -1, lastSliceHashCode: -1 01:51:49,507 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77] - expecting more 01:51:49,511 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-670: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77], sliceIndex=1, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123]] 01:51:49,511 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-670: getNextSlice: total size: 449, offset: 100, size: 100, index: 2 01:51:49,511 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-670: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77], data.length=100, sliceIndex=2, totalSlices=5, lastSliceHashCode=-1085826726, replyTo=Actor[akka://cluster-test/user/$a#1775345402]] 01:51:49,511 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77], data.length=100, sliceIndex=2, totalSlices=5, lastSliceHashCode=-1085826726, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1775345402]] 01:51:49,512 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77], sliceIndex: 2, lastSliceIndex: 1, assembledSize: 100, sliceHashCode: -1085826726, lastSliceHashCode: -1085826726 01:51:49,512 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77] - expecting more 01:51:49,512 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-670: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77], sliceIndex=2, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123]] 01:51:49,512 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-670: getNextSlice: total size: 449, offset: 200, size: 100, index: 3 01:51:49,512 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-670: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77], data.length=100, sliceIndex=3, totalSlices=5, lastSliceHashCode=-840689321, replyTo=Actor[akka://cluster-test/user/$a#1775345402]] 01:51:49,513 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77], data.length=100, sliceIndex=3, totalSlices=5, lastSliceHashCode=-840689321, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1775345402]] 01:51:49,513 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77], sliceIndex: 3, lastSliceIndex: 2, assembledSize: 200, sliceHashCode: -840689321, lastSliceHashCode: -840689321 01:51:49,513 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77] - expecting more 01:51:49,513 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-670: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77], sliceIndex=3, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123]] 01:51:49,513 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-670: getNextSlice: total size: 449, offset: 300, size: 100, index: 4 01:51:49,513 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-670: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77], data.length=100, sliceIndex=4, totalSlices=5, lastSliceHashCode=-382542389, replyTo=Actor[akka://cluster-test/user/$a#1775345402]] 01:51:49,514 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77], data.length=100, sliceIndex=4, totalSlices=5, lastSliceHashCode=-382542389, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1775345402]] 01:51:49,514 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77], sliceIndex: 4, lastSliceIndex: 3, assembledSize: 300, sliceHashCode: -382542389, lastSliceHashCode: -382542389 01:51:49,514 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77] - expecting more 01:51:49,514 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-670: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77], sliceIndex=4, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123]] 01:51:49,514 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-670: getNextSlice: total size: 449, offset: 400, size: 49, index: 5 01:51:49,514 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-670: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77], data.length=49, sliceIndex=5, totalSlices=5, lastSliceHashCode=32166009, replyTo=Actor[akka://cluster-test/user/$a#1775345402]] 01:51:49,515 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77], data.length=49, sliceIndex=5, totalSlices=5, lastSliceHashCode=32166009, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1775345402]] 01:51:49,515 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77], sliceIndex: 5, lastSliceIndex: 4, assembledSize: 400, sliceHashCode: 32166009, lastSliceHashCode: 32166009 01:51:49,515 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Received last slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77] 01:51:49,515 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Removing state for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77] 01:51:49,515 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: AssembledMessageState for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77] was removed from the cache due to EXPLICIT 01:51:49,516 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Notifying callback of re-assembled message Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1775345402], modifications=1}} 01:51:49,516 PM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testLargeReadReplySlicing: allocating new open transaction member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1 01:51:49,516 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-670: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77], sliceIndex=5, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123]] 01:51:49,516 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-670: Received last slice reply for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77] 01:51:49,516 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-670: Removing state for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77] 01:51:49,516 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-670: SlicedMessageState for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=670, messageId=77] was removed from the cache due to EXPLICIT 01:51:49,517 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1775345402], modifications=1} with Envelope{sessionId=0, txSequence=0, message=ModifyTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0}} 01:51:49,517 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@53d03aa1 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1775345402], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=863816824084} 01:51:49,517 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@53d03aa1 transmitted 1 requests 01:51:49,517 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1775345402], modifications=1}, enqueuedTicks=863816824084, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ModifyTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0}} 01:51:49,517 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1775345402], modifications=1} completed with ModifyTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0} 01:51:49,518 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-669: slice: identifier: member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1}} 01:51:49,518 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-669: Message size: 390 bytes, total slices to send: 4 01:51:49,518 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-669: getNextSlice: total size: 390, offset: 0, size: 100, index: 1 01:51:49,518 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-669: Sending first slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78], data.length=100, sliceIndex=1, totalSlices=4, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123]] 01:51:49,519 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78], data.length=100, sliceIndex=1, totalSlices=4, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123]] 01:51:49,519 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Received first slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78] - creating AssembledMessageState 01:51:49,519 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AssembledMessageState - member-2-frontend-datastore-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78], sliceIndex: 1, lastSliceIndex: 0, assembledSize: 0, sliceHashCode: -1, lastSliceHashCode: -1 01:51:49,519 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78] - expecting more 01:51:49,519 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-672: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78], sliceIndex=1, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1775345402]] 01:51:49,520 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-669: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78], sliceIndex=1, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1775345402]] 01:51:49,520 PM [cluster-test-shard-dispatcher-23] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-669: getNextSlice: total size: 390, offset: 100, size: 100, index: 2 01:51:49,520 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-669: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78], data.length=100, sliceIndex=2, totalSlices=4, lastSliceHashCode=-1569038920, replyTo=Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123]] 01:51:49,520 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78], data.length=100, sliceIndex=2, totalSlices=4, lastSliceHashCode=-1569038920, replyTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123]] 01:51:49,520 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AssembledMessageState - member-2-frontend-datastore-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78], sliceIndex: 2, lastSliceIndex: 1, assembledSize: 100, sliceHashCode: -1569038920, lastSliceHashCode: -1569038920 01:51:49,520 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78] - expecting more 01:51:49,520 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-672: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78], sliceIndex=2, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1775345402]] 01:51:49,521 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-669: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78], sliceIndex=2, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1775345402]] 01:51:49,521 PM [cluster-test-shard-dispatcher-23] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-669: getNextSlice: total size: 390, offset: 200, size: 100, index: 3 01:51:49,521 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-669: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78], data.length=100, sliceIndex=3, totalSlices=4, lastSliceHashCode=1439492931, replyTo=Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123]] 01:51:49,521 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78], data.length=100, sliceIndex=3, totalSlices=4, lastSliceHashCode=1439492931, replyTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123]] 01:51:49,521 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AssembledMessageState - member-2-frontend-datastore-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78], sliceIndex: 3, lastSliceIndex: 2, assembledSize: 200, sliceHashCode: 1439492931, lastSliceHashCode: 1439492931 01:51:49,521 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78] - expecting more 01:51:49,522 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-672: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78], sliceIndex=3, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1775345402]] 01:51:49,522 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-669: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78], sliceIndex=3, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1775345402]] 01:51:49,522 PM [cluster-test-shard-dispatcher-23] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-669: getNextSlice: total size: 390, offset: 300, size: 90, index: 4 01:51:49,523 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-669: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78], data.length=90, sliceIndex=4, totalSlices=4, lastSliceHashCode=-31040526, replyTo=Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123]] 01:51:49,526 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78], data.length=90, sliceIndex=4, totalSlices=4, lastSliceHashCode=-31040526, replyTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123]] 01:51:49,527 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AssembledMessageState - member-2-frontend-datastore-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78], sliceIndex: 4, lastSliceIndex: 3, assembledSize: 300, sliceHashCode: -31040526, lastSliceHashCode: -31040526 01:51:49,527 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Received last slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78] 01:51:49,527 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Removing state for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78] 01:51:49,527 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: AssembledMessageState for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78] was removed from the cache due to EXPLICIT 01:51:49,527 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Notifying callback of re-assembled message Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1}} 01:51:49,527 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1775345402], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1}} 01:51:49,527 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1775345402], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=863816824084, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1}} 01:51:49,527 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1} 01:51:49,528 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLargeReadReplySlicing, generation=0} 01:51:49,528 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Issuing gracefulStop to shard member-1-shard-cars-testLargeReadReplySlicing 01:51:49,528 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLargeReadReplySlicing - waiting on 1 shards 01:51:49,528 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: onShutDown 01:51:49,528 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Initiating leader transfer 01:51:49,528 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4ce2dbf8 01:51:49,528 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testLargeReadReplySlicing: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4ce2dbf8 01:51:49,528 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Attempting to transfer leadership 01:51:49,528 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLargeReadReplySlicing: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:49,528 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-672: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78], sliceIndex=4, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1775345402]] 01:51:49,528 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-669: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78], sliceIndex=4, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1775345402]] 01:51:49,528 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-669: Received last slice reply for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78] 01:51:49,528 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-669: Removing state for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78] 01:51:49,528 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-669: SlicedMessageState for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=669, messageId=78] was removed from the cache due to EXPLICIT 01:51:49,528 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLargeReadReplySlicing, leaderId=null, leaderPayloadVersion=13] 01:51:49,528 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:49,528 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:49,538 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLargeReadReplySlicing] 01:51:49,538 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing Received follower initial sync status for member-2-shard-cars-testLargeReadReplySlicing status sync done true 01:51:49,538 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLargeReadReplySlicing, leaderId=null, leaderPayloadVersion=13] 01:51:49,538 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:49,538 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:49,538 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@53d03aa1 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@651465d, splicing queue 01:51:49,538 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@53d03aa1 queue spliced 0 messages 01:51:49,538 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:49,538 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: refreshing backend for shard 1 01:51:49,539 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:49,539 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:49,539 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testLargeReadReplySlicing, matchIndex: -1, lastIndex: -1, isVoting: true 01:51:49,539 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:49,539 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Follower's log matches - sending ElectionTimeout 01:51:49,539 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1477913742] 01:51:49,539 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLargeReadReplySlicing: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:49,539 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Leader transfer complete 01:51:49,547 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:49,547 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testLargeReadReplySlicing, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:51:49,938 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: All shards shutdown - sending PoisonPill to self 01:51:49,938 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-testLargeReadReplySlicing: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:51:49,939 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-1-shard-cars-testLargeReadReplySlicing 01:51:49,939 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-672: Closing 01:51:49,939 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Aborting 0 pending queued transactions 01:51:49,939 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testLargeReadReplySlicing 01:51:49,940 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLargeReadReplySlicing: Closing 01:51:49,940 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLargeReadReplySlicing, generation=0} 01:51:49,940 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLargeReadReplySlicing: shutting down 01:51:49,940 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLargeReadReplySlicing: Closing 01:51:49,940 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-1052584595] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-1052584595] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:49,940 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-1052584595] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-1052584595] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:49,940 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#1008622233] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1008622233] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:49,940 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLargeReadReplySlicing: Closing 01:51:49,940 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-1052584595] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-1052584595] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:49,940 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#1008622233] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1008622233] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:49,940 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-1052584595] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-1052584595] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:49,941 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:49,941 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:49,945 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:49,945 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:49,945 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:49,945 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:49,947 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:51:49,960 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Closing 01:51:49,960 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testLargeReadReplySlicing 01:51:49,960 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testLargeReadReplySlicing (Follower): Closing 01:51:49,961 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: Aborting 0 pending queued transactions 01:51:49,961 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testLargeReadReplySlicing 01:51:49,961 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:49,962 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:50,954 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [6674873829856076223] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:50,982 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:50,982 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:50,983 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:51:50,984 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [3933556073897378099] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [03/11/2024 13:51:50.985] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:50,992 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:50,992 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:50,994 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:50.995] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:51,12 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:51,17 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [7215635008473551815] 01:51:51,17 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:51:51,18 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:51:51,18 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:51:51,22 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:51,22 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:51,29 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:51:51,29 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:51:51,29 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:51:51,34 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:51:51,41 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-3232233343804304326] 01:51:51,41 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:51:51,44 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:51,44 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:51:51,44 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:51,44 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:51,57 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:51:51,72 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:51:51,73 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:51,77 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-3768483139352796700] 01:51:51,77 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:51:51,78 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:51,78 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:51:51,79 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:51,79 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:51,89 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:51,90 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:51,91 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleTransactionsWritesInQuickSuccession 01:51:51,98 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 01:51:51,99 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1808634169] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:51,105 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: no initial generation override, starting from 0 01:51:51,105 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:51:51,105 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@49fa8d85 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2259ee88 01:51:51,105 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testSingleTransactionsWritesInQuickSuccession 01:51:51,105 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Creating local shard: member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:51,106 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:51,106 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:51,109 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@18fbd84f 01:51:51,109 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, 0, 1710165111105, None)) 01:51:51,111 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165111104,0,0)) 01:51:51,111 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2259ee88 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@b194eba 01:51:51,111 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} started 01:51:51,111 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6c0687f2 01:51:51,112 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:51,112 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:51,112 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:51,114 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:51,115 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:51,115 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:51,115 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: updated peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491] 01:51:51,115 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, persistent : true 01:51:51,115 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:51,115 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with journal batch size 1 01:51:51,115 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#332109294 created and ready for shard:member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:51,115 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleTransactionsWritesInQuickSuccession 01:51:51,116 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#332109294] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:51,121 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:51,121 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:51,122 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:51:51,123 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Initializing shard [cars] 01:51:51,123 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is initialized 01:51:51,124 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-1:snapshot-read#1549463570] 01:51:51,124 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-1:snapshot-read#1549463570] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:51,124 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:51,129 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 01:51:51,129 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 01:51:51,129 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from null to Follower 01:51:51,129 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , registered listener akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession 01:51:51,129 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from null to Follower 01:51:51,130 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-1162099069] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:51,131 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 01:51:51,132 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSingleTransactionsWritesInQuickSuccession 01:51:51,132 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: no initial generation override, starting from 0 01:51:51,132 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Creating local shard: member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:51,132 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:51:51,132 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@24cd055e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7b6a73c2 01:51:51,132 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:51,132 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:51,134 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4b422dfd 01:51:51,134 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, 0, 1710165111132, None)) 01:51:51,135 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165111131,0,0)) 01:51:51,135 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7b6a73c2 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@69b0db5d 01:51:51,135 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@57d0a558 01:51:51,135 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} started 01:51:51,135 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:51,135 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:51,135 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:51,135 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:51,136 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: updated peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession#1903853756] 01:51:51,136 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, persistent : true 01:51:51,136 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession#1903853756] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:51,136 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#31208534 created and ready for shard:member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:51,136 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-2:snapshot-read#1105328408] 01:51:51,136 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#31208534] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:51,136 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-2:snapshot-read#1105328408] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:51,136 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with journal batch size 1 01:51:51,137 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:51,137 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:51,137 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Initializing shard [cars] 01:51:51,137 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession is initialized 01:51:51,137 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 01:51:51,137 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 01:51:51,137 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:51,137 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from null to Follower 01:51:51,137 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession , registered listener akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession 01:51:51,137 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession from null to Follower 01:51:51,185 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1808634169] 01:51:51,186 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: findLocalShard : cars 01:51:51,186 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491] 01:51:51,427 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:51,427 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Received ElectionTimeout - switching to Candidate 01:51:51,427 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] 01:51:51,427 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:51,427 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 1 01:51:51,427 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:51,427 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Closing 01:51:51,427 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 01:51:51,427 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 01:51:51,427 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from Follower to Candidate 01:51:51,427 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from Follower to Candidate 01:51:51,647 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 01:51:51,647 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:51,647 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 2 01:51:51,728 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:51,897 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 01:51:51,897 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:51,897 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 3 01:51:52,38 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:51:52,38 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:51:52,39 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:52,39 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,39 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,39 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:52,39 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,39 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,39 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,44 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:52,44 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,44 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,44 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:52,44 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,44 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,96 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:51:52,196 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 01:51:52,196 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,196 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 4 01:51:52,196 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,198 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:52,198 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Cluster state: [] 01:51:52,198 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:52,198 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:52,198 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=null 01:51:52,198 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:52,198 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,198 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:52,199 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:52,199 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:52,199 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] 01:51:52,200 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:52,200 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:52,200 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: true, hasLeader : true 01:51:52,200 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: became leader with frontend state for [] 01:51:52,200 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from Candidate to Leader 01:51:52,200 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderPayloadVersion=13] 01:51:52,200 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:52,200 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:52,200 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from Candidate to Leader 01:51:52,200 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All Shards are ready - data store testSingleTransactionsWritesInQuickSuccession is ready 01:51:52,200 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Closing 01:51:52,201 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: getLeaderAddress leaderId = member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,201 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderPayloadVersion=13] 01:51:52,201 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:52,201 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:52,201 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All Shards are ready - data store testSingleTransactionsWritesInQuickSuccession is ready 01:51:52,201 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession Received follower initial sync status for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession status sync done false 01:51:52,201 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: -1, nextIndex: 0 01:51:52,201 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:52,244 PM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@af92f3d 01:51:52,244 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@af92f3d changed state from IDLE to TX_OPEN 01:51:52,244 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} 01:51:52,244 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:52,244 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:52,244 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-1162099069] 01:51:52,244 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:52,244 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} shard 1 01:51:52,244 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, primaryVersion=13] 01:51:52,244 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:51:52,244 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:51:52,244 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, primaryVersion=13] 01:51:52,244 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 01:51:52,244 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@4b277697, attempting to connect 01:51:52,244 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@27214d4b 01:51:52,244 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@af92f3d changed state from TX_OPEN to IDLE 01:51:52,244 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@af92f3d readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 01:51:52,245 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 01:51:52,245 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@7703abde 01:51:52,245 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} is not yet known 01:51:52,246 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, nanosAgo=10720, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:51:52,247 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:52,247 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:52,247 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:52,247 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@57da5b03 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@96a341c 01:51:52,247 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:51:52,247 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:52,247 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866575150209} 01:51:52,247 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@96a341c 01:51:52,247 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:51:52,247 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} 01:51:52,247 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=2, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} 01:51:52,247 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 01:51:52,247 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2238/0x00000008016aba80@4669645c 01:51:52,247 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=2, protocol=READY}, enqueuedTicks=866575445516} 01:51:52,247 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=2, protocol=READY}} 01:51:52,248 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message size: 415 bytes, total slices to send: 1 01:51:52,248 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message does not need to be sliced - sending original message 01:51:52,248 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED} 01:51:52,248 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 1 01:51:52,248 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2277/0x00000008016c5c10@38923143 01:51:52,248 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE}, enqueuedTicks=866575609870} 01:51:52,248 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE}} 01:51:52,248 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message size: 295 bytes, total slices to send: 1 01:51:52,248 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message does not need to be sliced - sending original message 01:51:52,248 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@af92f3d finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:52,248 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 01:51:52,248 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@3509a9e0 01:51:52,248 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5a485abb superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1562a08b, splicing queue 01:51:52,248 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5a485abb queue spliced 0 messages 01:51:52,248 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.611 ms 01:51:52,249 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 01:51:52,249 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 01:51:52,249 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:51:52,249 PM [cluster-test-shard-dispatcher-29] [DEBUG] LeaderFrontendState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:51:52,249 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:51:52,249 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:51:52,249 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@7ea88432 01:51:52,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:51:52,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 ready 01:51:52,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 01:51:52,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 to ready 01:51:52,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 is already in state READY (READY) 01:51:52,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 initiating direct canCommit 01:51:52,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:51:52,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 validated 01:51:52,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:52,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 initiating direct preCommit 01:51:52,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:51:52,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 candidate ready 01:51:52,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1: canCommit - candidate rootPath: / 01:51:52,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1: canCommit - messages: [] 01:51:52,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1: preCommit - successfulFromPrevious: [] 01:51:52,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 prepared 01:51:52,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:52,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 initiating direct commit 01:51:52,250 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:51:52,251 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 184 01:51:52,251 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 01:51:52,251 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 01:51:52,251 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:52,251 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:51:52,251 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,251 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}]]] 01:51:52,251 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 submitted to persistence 01:51:52,251 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 01:51:52,251 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 01:51:52,251 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=2, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0}} 01:51:52,251 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=2, protocol=READY}, enqueuedTicks=866575445516, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0}} 01:51:52,251 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=2, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0} 01:51:52,252 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}]]] 01:51:52,252 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 01:51:52,252 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:52,252 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75} 01:51:52,252 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 01:51:52,252 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=184} 01:51:52,252 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 01:51:52,252 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 01:51:52,252 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:52,252 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 01:51:52,252 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 01:51:52,252 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession Received follower initial sync status for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession status sync done true 01:51:52,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 1, nextIndex: 2 01:51:52,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 01:51:52,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 0 01:51:52,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from -1 to 0 01:51:52,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75} 01:51:52,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2241/0x00000008016af5d0@316cacf9 01:51:52,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] LeaderFrontendState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:51:52,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@a15a865}, purged=MutableUnsignedLongSet{size=0}} 01:51:52,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:51:52,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 1 01:51:52,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 0 to 1 01:51:52,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 1 data Simple{size=184} 01:51:52,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:51:52,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:51:52,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1: commit - successfulFromPrevious: [] 01:51:52,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:51:52,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 01:51:52,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#833758814] with name member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher 01:51:52,255 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#833758814] 01:51:52,255 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 0 for term 4 01:51:52,256 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:52,256 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:51:52,256 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=1 01:51:52,256 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#833758814] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:52,256 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 81.94 μs 01:51:52,256 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 1 01:51:52,256 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 01:51:52,256 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 0 01:51:52,256 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from -1 to 0 01:51:52,256 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75} 01:51:52,256 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@35bf5653}, purged=MutableUnsignedLongSet{size=0}} 01:51:52,256 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:51:52,256 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 1 01:51:52,256 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 0 to 1 01:51:52,256 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 1 data Simple{size=184} 01:51:52,257 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:51:52,257 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:51:52,257 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#702778672] with name member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher 01:51:52,257 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 0 for term 4 01:51:52,257 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=1 01:51:52,257 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0}} 01:51:52,257 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866575150209, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0}} 01:51:52,257 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0} 01:51:52,257 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1}} 01:51:52,257 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#702778672] 01:51:52,257 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE}, enqueuedTicks=866575609870, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1}} 01:51:52,257 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE} completed with TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1} 01:51:52,257 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} directCommit completed 01:51:52,257 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:51:52,257 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: initiating purge 01:51:52,257 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:51:52,257 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@552bb37d 01:51:52,257 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} 01:51:52,257 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED}: initiating purge 01:51:52,257 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 51.36 μs 01:51:52,258 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#702778672] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:52,257 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 2 01:51:52,258 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@af92f3d changed state from IDLE to TX_OPEN 01:51:52,258 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@4d4c49d3 01:51:52,258 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:52,258 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866588251953} 01:51:52,258 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:51:52,258 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 0 01:51:52,258 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@22109a27 01:51:52,258 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY}, enqueuedTicks=866589103034} 01:51:52,258 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY}} 01:51:52,258 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message size: 455 bytes, total slices to send: 1 01:51:52,258 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message does not need to be sliced - sending original message 01:51:52,258 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@af92f3d changed state from TX_OPEN to IDLE 01:51:52,258 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@af92f3d readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 01:51:52,258 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=FLUSHED} allocated sequence 1 01:51:52,258 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@2871a8f2 01:51:52,258 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE}, enqueuedTicks=866589465873} 01:51:52,258 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE}} 01:51:52,258 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message size: 296 bytes, total slices to send: 1 01:51:52,258 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message does not need to be sliced - sending original message 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@3285e02f 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 ready 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}], operation=WRITE}}}}}}}]} 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 to ready 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 is already in state READY (READY) 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 initiating direct canCommit 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 validated 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 initiating direct preCommit 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 candidate ready 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1: canCommit - candidate rootPath: / 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1: canCommit - messages: [] 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1: preCommit - successfulFromPrevious: [] 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 prepared 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 initiating direct commit 01:51:52,262 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:51:52,263 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 01:51:52,263 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 01:51:52,263 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 01:51:52,263 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:52,263 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 01:51:52,263 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,263 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}]]] 01:51:52,263 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 submitted to persistence 01:51:52,263 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 01:51:52,263 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 01:51:52,264 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=4, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0}} 01:51:52,264 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY}, enqueuedTicks=866589103034, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0}} 01:51:52,264 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0} 01:51:52,264 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}]]] 01:51:52,264 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 01:51:52,264 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:51:52,264 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76} 01:51:52,264 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 01:51:52,264 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 01:51:52,264 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 01:51:52,264 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 01:51:52,264 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:52,265 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 01:51:52,265 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 01:51:52,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 3, nextIndex: 4 01:51:52,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 01:51:52,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 2 01:51:52,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 1 to 2 01:51:52,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76} 01:51:52,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@23690f42 01:51:52,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:51:52,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:51:52,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 3 01:51:52,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 2 to 3 01:51:52,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 3 data Simple{size=241} 01:51:52,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:51:52,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:51:52,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1: commit - successfulFromPrevious: [] 01:51:52,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 01:51:52,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:51:52,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 2 for term 4 01:51:52,266 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:51:52,267 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=3 01:51:52,267 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:51:52,268 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 79.41 μs 01:51:52,268 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 3 01:51:52,268 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 01:51:52,268 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2}} 01:51:52,268 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 2 01:51:52,268 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 1 to 2 01:51:52,268 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866588251953, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2}} 01:51:52,268 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76} 01:51:52,268 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 01:51:52,268 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@96a341c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=DONE} 01:51:52,268 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: purge completed 01:51:52,268 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@57da5b03 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 01:51:52,268 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=5, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1}} 01:51:52,268 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE}, enqueuedTicks=866589465873, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1}} 01:51:52,268 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 01:51:52,268 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 3 01:51:52,268 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=FLUSHED} directCommit completed 01:51:52,268 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 2 to 3 01:51:52,268 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=FLUSHED}: initiating purge 01:51:52,268 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 3 data Simple{size=241} 01:51:52,268 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=DONE} allocated sequence 2 01:51:52,268 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@4aa4a1f 01:51:52,268 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:52,268 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866599411020} 01:51:52,269 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 01:51:52,269 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 0 01:51:52,269 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:51:52,269 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@66512722 01:51:52,269 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=866599736568} 01:51:52,269 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:51:52,269 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:51:52,269 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 2 for term 4 01:51:52,269 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 45.41 μs 01:51:52,269 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=3 01:51:52,272 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:51:52,272 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 01:51:52,272 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 01:51:52,272 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:52,272 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 01:51:52,272 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:51:52,272 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@199988fc 01:51:52,272 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 01:51:52,272 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0}} 01:51:52,272 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 388 bytes, total slices to send: 1 01:51:52,272 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 01:51:52,273 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0}} 01:51:52,273 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=866599736568, sessionId=0, txSequence=7} with Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0}} 01:51:52,273 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0} 01:51:52,273 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@96a341c aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} 01:51:52,273 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 1 01:51:52,274 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@51b0ba44 01:51:52,274 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT}, enqueuedTicks=866604582838} 01:51:52,274 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT}} 01:51:52,274 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message size: 296 bytes, total slices to send: 1 01:51:52,274 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message does not need to be sliced - sending original message 01:51:52,274 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@af92f3d changed state from IDLE to TX_OPEN 01:51:52,274 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:51:52,274 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=OPEN} allocated sequence 0 01:51:52,274 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@3d4e1185 01:51:52,274 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY}, enqueuedTicks=866605320746} 01:51:52,274 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, message: Envelope{sessionId=0, txSequence=9, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY}} 01:51:52,274 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message size: 455 bytes, total slices to send: 1 01:51:52,274 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message does not need to be sliced - sending original message 01:51:52,275 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@af92f3d changed state from TX_OPEN to IDLE 01:51:52,275 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@af92f3d readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 01:51:52,275 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=FLUSHED} allocated sequence 1 01:51:52,275 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@2d7cba83 01:51:52,275 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE}, enqueuedTicks=866605730386} 01:51:52,275 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, message: Envelope{sessionId=0, txSequence=a, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE}} 01:51:52,275 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message size: 296 bytes, total slices to send: 1 01:51:52,275 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message does not need to be sliced - sending original message 01:51:52,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 01:51:52,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 01:51:52,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 01:51:52,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 01:51:52,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:51:52,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:51:52,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@199988fc 01:51:52,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:51:52,276 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 ready 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}], operation=WRITE}}}}}}}]} 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 to ready 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 is already in state READY (READY) 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 initiating direct canCommit 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 validated 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 initiating direct preCommit 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 candidate ready 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1: canCommit - candidate rootPath: / 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1: canCommit - messages: [] 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1: preCommit - successfulFromPrevious: [] 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 prepared 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 initiating direct commit 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 4, leaderLastIndex: 6, leaderSnapShotIndex: 2 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}], SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}], SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}]]] 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 submitted to persistence 01:51:52,277 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 01:51:52,278 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0}} 01:51:52,278 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY}, enqueuedTicks=866605320746, sessionId=0, txSequence=9} with Envelope{sessionId=0, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0}} 01:51:52,278 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0} 01:51:52,278 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 01:51:52,280 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}], SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}], SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}]]] 01:51:52,280 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 3 01:51:52,280 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 01:51:52,280 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77} 01:51:52,280 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 01:51:52,280 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 01:51:52,280 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 01:51:52,280 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 01:51:52,280 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 01:51:52,280 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 4 01:51:52,280 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:52,280 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 01:51:52,280 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 01:51:52,280 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 6, nextIndex: 7 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 3 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 4 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 3 to 4 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77} 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@48327c12 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 5 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 4 to 5 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 5 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2284/0x00000008016d86d0@5066d17a 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 6 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 5 to 6 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 6 data Simple{size=241} 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1: commit - successfulFromPrevious: [] 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 changing state from COMMIT_PENDING to COMMITTED 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:51:52,281 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} with Envelope{sessionId=0, txSequence=6, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2}} 01:51:52,281 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866599411020, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2}} 01:51:52,281 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=DONE}: purge completed 01:51:52,281 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@96a341c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=DONE} 01:51:52,281 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 5 for term 4 01:51:52,281 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1}} 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 01:51:52,281 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT}, enqueuedTicks=866604582838, sessionId=0, txSequence=8} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1}} 01:51:52,281 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 74.54 μs 01:51:52,281 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1} 01:51:52,281 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN}: initiating purge 01:51:52,281 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=DONE} allocated sequence 2 01:51:52,281 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=6 01:51:52,281 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@715d594b 01:51:52,281 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866612489474} 01:51:52,282 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 6 01:51:52,282 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 01:51:52,282 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 3, lastIndex(): 6 01:51:52,282 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 4 01:51:52,282 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 3 to 4 01:51:52,282 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77} 01:51:52,282 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 01:51:52,282 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 01:51:52,282 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 01:51:52,282 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 5 01:51:52,282 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:52,282 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 4 to 5 01:51:52,282 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 5 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 01:51:52,282 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 01:51:52,283 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 6 01:51:52,283 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 5 to 6 01:51:52,283 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 6 data Simple{size=241} 01:51:52,283 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:51:52,283 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1}} 01:51:52,283 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE}, enqueuedTicks=866605730386, sessionId=0, txSequence=10} with Envelope{sessionId=0, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1}} 01:51:52,283 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 01:51:52,283 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=FLUSHED} directCommit completed 01:51:52,283 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=FLUSHED}: initiating purge 01:51:52,283 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=DONE} allocated sequence 2 01:51:52,283 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@79e5bc46 01:51:52,283 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866613848397} 01:51:52,283 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:51:52,283 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 5 for term 4 01:51:52,283 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 01:51:52,283 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} allocated sequence 0 01:51:52,283 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=6 01:51:52,283 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@2b6f5aeb 01:51:52,283 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=866614152825} 01:51:52,283 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:51:52,283 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 52.59 μs 01:51:52,288 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 7 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,288 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}]]] 01:51:52,288 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:51:52,289 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 01:51:52,289 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 01:51:52,289 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, logIndex: 8, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:52,289 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 01:51:52,289 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:51:52,289 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@132e42e6 01:51:52,289 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, message: Envelope{sessionId=0, txSequence=d, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0}} 01:51:52,289 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 408 bytes, total slices to send: 1 01:51:52,289 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 01:51:52,289 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}]]] 01:51:52,289 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 01:51:52,289 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:51:52,289 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 01:51:52,289 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 01:51:52,289 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 01:51:52,289 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 01:51:52,289 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=7, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:52,290 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 01:51:52,290 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=d, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0}} 01:51:52,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 7, nextIndex: 8 01:51:52,290 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=866614152825, sessionId=0, txSequence=13} with Envelope{sessionId=0, txSequence=d, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0}} 01:51:52,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 6 01:51:52,290 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0} 01:51:52,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 7 01:51:52,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 6 to 7 01:51:52,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 7 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 01:51:52,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@768ad80 01:51:52,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 01:51:52,290 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@96a341c aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} 01:51:52,290 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} allocated sequence 1 01:51:52,290 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@4ef37f46 01:51:52,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 01:51:52,290 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT}, enqueuedTicks=866621315832} 01:51:52,290 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, message: Envelope{sessionId=0, txSequence=e, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT}} 01:51:52,290 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message size: 296 bytes, total slices to send: 1 01:51:52,290 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message does not need to be sliced - sending original message 01:51:52,291 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@af92f3d changed state from IDLE to TX_OPEN 01:51:52,291 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:51:52,291 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=OPEN} allocated sequence 0 01:51:52,291 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@45cb7e8a 01:51:52,291 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY}, enqueuedTicks=866621783124} 01:51:52,291 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, message: Envelope{sessionId=0, txSequence=f, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY}} 01:51:52,291 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message size: 455 bytes, total slices to send: 1 01:51:52,291 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message does not need to be sliced - sending original message 01:51:52,291 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@af92f3d changed state from TX_OPEN to IDLE 01:51:52,291 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@af92f3d readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 01:51:52,291 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=FLUSHED} allocated sequence 1 01:51:52,291 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@628c9281 01:51:52,291 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE}, enqueuedTicks=866622030550} 01:51:52,291 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, message: Envelope{sessionId=0, txSequence=10, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE}} 01:51:52,291 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message size: 296 bytes, total slices to send: 1 01:51:52,291 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message does not need to be sliced - sending original message 01:51:52,290 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 6 for term 4 01:51:52,291 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 8, leaderLastIndex: 8, leaderSnapShotIndex: 6 01:51:52,291 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 8 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,291 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=7, prevLogTerm=4, leaderCommit=7, replicatedToAllIndex=6, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}]]] 01:51:52,291 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} with Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2}} 01:51:52,291 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866612489474, sessionId=0, txSequence=11} with Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2}} 01:51:52,291 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=7 01:51:52,292 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=DONE}: purge completed 01:51:52,292 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@96a341c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=DONE} 01:51:52,292 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=7, prevLogTerm=4, leaderCommit=7, replicatedToAllIndex=6, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}]]] 01:51:52,292 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:51:52,292 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 7, entries to be added from: 0 01:51:52,292 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77} 01:51:52,292 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 01:51:52,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 01:51:52,292 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 01:51:52,292 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 7 01:51:52,292 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=8, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:52,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 01:51:52,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 01:51:52,292 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 7, context.getLastApplied(): 6, lastIndex(): 8 01:51:52,292 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 7 01:51:52,292 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 6 to 7 01:51:52,292 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 7 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 01:51:52,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, logIndex: 9, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 01:51:52,292 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 01:51:52,292 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 6 for term 4 01:51:52,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:51:52,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:51:52,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@132e42e6 01:51:52,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:51:52,292 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 ready 01:51:52,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 01:51:52,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=7 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}], operation=WRITE}}}}}}}]} 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 to ready 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 is already in state READY (READY) 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 initiating direct canCommit 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 validated 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 initiating direct preCommit 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 candidate ready 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1: canCommit - candidate rootPath: / 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1: canCommit - messages: [] 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1: preCommit - successfulFromPrevious: [] 01:51:52,293 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=f, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0}} 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 prepared 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:52,293 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY}, enqueuedTicks=866621783124, sessionId=0, txSequence=15} with Envelope{sessionId=0, txSequence=f, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0}} 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 initiating direct commit 01:51:52,293 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0} 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 01:51:52,293 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, logIndex: 10, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:52,294 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 8, leaderLastIndex: 10, leaderSnapShotIndex: 6 01:51:52,294 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 8 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,294 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 submitted to persistence 01:51:52,294 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 01:51:52,294 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 01:51:52,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 8, nextIndex: 9 01:51:52,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 8, lastAppliedIndex: 7 01:51:52,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 8 01:51:52,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 7 to 8 01:51:52,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 8 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77} 01:51:52,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@3a81f143 01:51:52,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:51:52,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:51:52,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 7 for term 4 01:51:52,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 9, leaderLastIndex: 10, leaderSnapShotIndex: 7 01:51:52,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 9 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=8, prevLogTerm=4, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}], SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}]]] 01:51:52,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=8 01:51:52,298 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} with Envelope{sessionId=0, txSequence=c, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2}} 01:51:52,298 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866613848397, sessionId=0, txSequence=12} with Envelope{sessionId=0, txSequence=c, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2}} 01:51:52,298 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=DONE}: purge completed 01:51:52,298 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@96a341c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=DONE} 01:51:52,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=8, prevLogTerm=4, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}], SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}]]] 01:51:52,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 01:51:52,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 8, entries to be added from: 0 01:51:52,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 01:51:52,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 01:51:52,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 01:51:52,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 01:51:52,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 4 01:51:52,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 8 01:51:52,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=10, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:52,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 8, context.getLastApplied(): 7, lastIndex(): 10 01:51:52,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 8 01:51:52,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 7 to 8 01:51:52,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 8 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77} 01:51:52,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 01:51:52,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 7 for term 4 01:51:52,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 01:51:52,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 01:51:52,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=8 01:51:52,299 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 10, nextIndex: 11 01:51:52,299 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 10, lastAppliedIndex: 8 01:51:52,299 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 9 01:51:52,299 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 8 to 9 01:51:52,299 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 9 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 01:51:52,299 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2284/0x00000008016d86d0@5ace4709 01:51:52,299 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 01:51:52,299 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 10 01:51:52,299 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 9 to 10 01:51:52,299 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 10 data Simple{size=241} 01:51:52,299 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:51:52,299 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:51:52,299 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1: commit - successfulFromPrevious: [] 01:51:52,299 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 changing state from COMMIT_PENDING to COMMITTED 01:51:52,299 PM [cluster-test-shard-dispatcher-29] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:51:52,299 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:51:52,299 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 9 for term 4 01:51:52,299 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 11, leaderLastIndex: 10, leaderSnapShotIndex: 9 01:51:52,299 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=10 01:51:52,299 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 57.63 μs 01:51:52,299 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=e, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1}} 01:51:52,299 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT}, enqueuedTicks=866621315832, sessionId=0, txSequence=14} with Envelope{sessionId=0, txSequence=e, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1}} 01:51:52,300 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1} 01:51:52,300 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN}: initiating purge 01:51:52,300 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=DONE} allocated sequence 2 01:51:52,300 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@7de41c6 01:51:52,300 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866630561852} 01:51:52,300 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 01:51:52,300 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 01:51:52,300 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 01:51:52,300 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, logIndex: 11, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:52,300 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 01:51:52,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 10 01:51:52,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 10, context.getLastApplied(): 8, lastIndex(): 10 01:51:52,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 9 01:51:52,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 8 to 9 01:51:52,300 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=10, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1}} 01:51:52,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 9 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 01:51:52,300 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE}, enqueuedTicks=866622030550, sessionId=0, txSequence=16} with Envelope{sessionId=0, txSequence=10, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1}} 01:51:52,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 01:51:52,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 10 01:51:52,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 9 to 10 01:51:52,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 10 data Simple{size=241} 01:51:52,300 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=FLUSHED} directCommit completed 01:51:52,300 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=FLUSHED}: initiating purge 01:51:52,300 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=DONE} allocated sequence 2 01:51:52,300 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@7f60ae74 01:51:52,300 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:51:52,300 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866631366041} 01:51:52,300 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 01:51:52,301 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} allocated sequence 0 01:51:52,301 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@69327800 01:51:52,301 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=866631661479} 01:51:52,301 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:51:52,301 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:51:52,301 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 9 for term 4 01:51:52,301 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=10 01:51:52,301 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 60.05 μs 01:51:52,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 11 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=10, prevLogTerm=4, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}]]] 01:51:52,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:51:52,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 01:51:52,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 01:51:52,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, logIndex: 12, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:52,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 01:51:52,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:51:52,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@49d365e5 01:51:52,302 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, message: Envelope{sessionId=0, txSequence=13, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0}} 01:51:52,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 01:51:52,302 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 431 bytes, total slices to send: 1 01:51:52,302 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 01:51:52,302 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=10, prevLogTerm=4, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}]]] 01:51:52,302 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:51:52,302 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 10, entries to be added from: 0 01:51:52,302 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 01:51:52,302 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 01:51:52,302 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 01:51:52,302 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=11, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:52,302 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 01:51:52,303 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 11, nextIndex: 12 01:51:52,303 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 11, lastAppliedIndex: 10 01:51:52,303 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 11 01:51:52,303 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 10 to 11 01:51:52,303 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 11 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 01:51:52,303 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@cb33bb9 01:51:52,303 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 01:51:52,303 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 01:51:52,303 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 10 for term 4 01:51:52,303 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 12, leaderLastIndex: 12, leaderSnapShotIndex: 10 01:51:52,303 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 12 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,303 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=11, prevLogTerm=4, leaderCommit=11, replicatedToAllIndex=10, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}]]] 01:51:52,303 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=11 01:51:52,303 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=13, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0}} 01:51:52,303 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=866631661479, sessionId=0, txSequence=19} with Envelope{sessionId=0, txSequence=13, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0}} 01:51:52,303 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0} 01:51:52,303 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@96a341c aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} 01:51:52,303 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} allocated sequence 1 01:51:52,303 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@28deca3f 01:51:52,304 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT}, enqueuedTicks=866634556470} 01:51:52,304 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, message: Envelope{sessionId=0, txSequence=14, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT}} 01:51:52,304 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message size: 296 bytes, total slices to send: 1 01:51:52,304 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message does not need to be sliced - sending original message 01:51:52,304 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@af92f3d changed state from IDLE to TX_OPEN 01:51:52,304 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:51:52,304 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=OPEN} allocated sequence 0 01:51:52,304 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@16148dc4 01:51:52,304 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY}, enqueuedTicks=866634937480} 01:51:52,304 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, message: Envelope{sessionId=0, txSequence=15, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY}} 01:51:52,304 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message size: 455 bytes, total slices to send: 1 01:51:52,304 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message does not need to be sliced - sending original message 01:51:52,304 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@af92f3d changed state from TX_OPEN to IDLE 01:51:52,304 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@af92f3d readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 01:51:52,304 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=FLUSHED} allocated sequence 1 01:51:52,304 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@63fe339f 01:51:52,304 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE}, enqueuedTicks=866635180346} 01:51:52,304 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, message: Envelope{sessionId=0, txSequence=16, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE}} 01:51:52,304 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message size: 296 bytes, total slices to send: 1 01:51:52,304 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message does not need to be sliced - sending original message 01:51:52,305 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=11, prevLogTerm=4, leaderCommit=11, replicatedToAllIndex=10, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}]]] 01:51:52,305 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:51:52,305 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 11, entries to be added from: 0 01:51:52,305 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77} 01:51:52,305 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} with Envelope{sessionId=0, txSequence=11, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2}} 01:51:52,305 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 01:51:52,305 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866630561852, sessionId=0, txSequence=17} with Envelope{sessionId=0, txSequence=11, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2}} 01:51:52,305 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 01:51:52,305 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=DONE}: purge completed 01:51:52,305 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 11 01:51:52,305 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@96a341c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=DONE} 01:51:52,305 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=12, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:52,305 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 11, context.getLastApplied(): 10, lastIndex(): 12 01:51:52,305 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 11 01:51:52,305 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 10 to 11 01:51:52,305 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 11 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 01:51:52,305 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 01:51:52,305 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 10 for term 4 01:51:52,305 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 01:51:52,305 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 01:51:52,305 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 01:51:52,305 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, logIndex: 13, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 01:51:52,305 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 12, nextIndex: 13 01:51:52,305 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 12, lastAppliedIndex: 11 01:51:52,305 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 12 01:51:52,305 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 11 to 12 01:51:52,305 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 12 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77} 01:51:52,305 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@586f050b 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 11 for term 4 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 13, leaderLastIndex: 13, leaderSnapShotIndex: 11 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 13 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=12, prevLogTerm=4, leaderCommit=12, replicatedToAllIndex=11, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}]]] 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=12 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@49d365e5 01:51:52,306 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} with Envelope{sessionId=0, txSequence=12, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2}} 01:51:52,306 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866631366041, sessionId=0, txSequence=18} with Envelope{sessionId=0, txSequence=12, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2}} 01:51:52,306 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=DONE}: purge completed 01:51:52,306 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@96a341c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=DONE} 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 ready 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}], operation=WRITE}}}}}}}]} 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 to ready 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 is already in state READY (READY) 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 initiating direct canCommit 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 validated 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 initiating direct preCommit 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:51:52,306 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 candidate ready 01:51:52,307 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1: canCommit - candidate rootPath: / 01:51:52,307 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1: canCommit - messages: [] 01:51:52,307 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1: preCommit - successfulFromPrevious: [] 01:51:52,307 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 prepared 01:51:52,307 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:52,307 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 initiating direct commit 01:51:52,307 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:51:52,307 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 01:51:52,307 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 01:51:52,307 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 01:51:52,307 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, logIndex: 14, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:52,307 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 13, leaderLastIndex: 14, leaderSnapShotIndex: 11 01:51:52,307 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 13 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,307 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 submitted to persistence 01:51:52,307 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 01:51:52,307 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 01:51:52,307 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=11 01:51:52,307 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=12, prevLogTerm=4, leaderCommit=12, replicatedToAllIndex=11, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}]]] 01:51:52,308 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:51:52,307 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=15, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0}} 01:51:52,308 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 12, entries to be added from: 0 01:51:52,308 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 01:51:52,308 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 01:51:52,308 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY}, enqueuedTicks=866634937480, sessionId=0, txSequence=21} with Envelope{sessionId=0, txSequence=15, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0}} 01:51:52,308 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0} 01:51:52,308 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 01:51:52,308 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 12 01:51:52,308 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=13, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:52,308 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 12, context.getLastApplied(): 11, lastIndex(): 13 01:51:52,308 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 12 01:51:52,308 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 11 to 12 01:51:52,308 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 12 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77} 01:51:52,308 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 01:51:52,308 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 11 for term 4 01:51:52,308 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 01:51:52,308 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 01:51:52,308 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=12 01:51:52,308 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 13, nextIndex: 14 01:51:52,308 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 13, lastAppliedIndex: 12 01:51:52,308 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 13 01:51:52,308 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 12 to 13 01:51:52,308 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 13 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 01:51:52,308 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2284/0x00000008016d86d0@7bd8b263 01:51:52,309 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 01:51:52,309 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 12 for term 4 01:51:52,309 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 14, leaderLastIndex: 14, leaderSnapShotIndex: 12 01:51:52,309 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 14 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,309 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=13, prevLogTerm=4, leaderCommit=13, replicatedToAllIndex=12, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}]]] 01:51:52,309 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=13 01:51:52,309 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=14, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1}} 01:51:52,309 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT}, enqueuedTicks=866634556470, sessionId=0, txSequence=20} with Envelope{sessionId=0, txSequence=14, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1}} 01:51:52,309 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1} 01:51:52,309 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN}: initiating purge 01:51:52,309 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=DONE} allocated sequence 2 01:51:52,309 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@67e5aea1 01:51:52,309 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866640450387} 01:51:52,310 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=13, prevLogTerm=4, leaderCommit=13, replicatedToAllIndex=12, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}]]] 01:51:52,310 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:51:52,310 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 13, entries to be added from: 0 01:51:52,310 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 01:51:52,310 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 01:51:52,310 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 01:51:52,310 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 13 01:51:52,310 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=14, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:52,310 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 13, context.getLastApplied(): 12, lastIndex(): 14 01:51:52,310 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 13 01:51:52,310 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 12 to 13 01:51:52,310 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 13 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 01:51:52,310 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 01:51:52,310 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 12 for term 4 01:51:52,310 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 01:51:52,310 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=13 01:51:52,310 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 01:51:52,310 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 01:51:52,310 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 01:51:52,310 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, logIndex: 15, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:52,310 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 14, nextIndex: 15 01:51:52,310 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 14, lastAppliedIndex: 13 01:51:52,310 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 14 01:51:52,310 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 13 to 14 01:51:52,310 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 14 data Simple{size=241} 01:51:52,310 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:51:52,310 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:51:52,310 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1: commit - successfulFromPrevious: [] 01:51:52,310 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 changing state from COMMIT_PENDING to COMMITTED 01:51:52,311 PM [cluster-test-shard-dispatcher-29] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:51:52,311 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:51:52,311 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 13 for term 4 01:51:52,311 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 15, leaderLastIndex: 15, leaderSnapShotIndex: 13 01:51:52,311 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 77.85 μs 01:51:52,311 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 15 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,311 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=14, prevLogTerm=4, leaderCommit=14, replicatedToAllIndex=13, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}]]] 01:51:52,311 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=14 01:51:52,311 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 01:51:52,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=16, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1}} 01:51:52,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE}, enqueuedTicks=866635180346, sessionId=0, txSequence=22} with Envelope{sessionId=0, txSequence=16, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1}} 01:51:52,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=FLUSHED} directCommit completed 01:51:52,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=FLUSHED}: initiating purge 01:51:52,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=DONE} allocated sequence 2 01:51:52,311 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@67832c2e 01:51:52,312 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866642516848} 01:51:52,312 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 01:51:52,312 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} allocated sequence 0 01:51:52,312 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@744803f1 01:51:52,312 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=866642763344} 01:51:52,312 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=14, prevLogTerm=4, leaderCommit=14, replicatedToAllIndex=13, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}]]] 01:51:52,312 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:51:52,312 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 14, entries to be added from: 0 01:51:52,312 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 01:51:52,312 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 01:51:52,312 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 01:51:52,312 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 14 01:51:52,312 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=15, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:52,312 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 14, context.getLastApplied(): 13, lastIndex(): 15 01:51:52,312 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 14 01:51:52,312 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 13 to 14 01:51:52,312 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 14 data Simple{size=241} 01:51:52,312 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:51:52,312 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:51:52,312 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:51:52,312 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 13 for term 4 01:51:52,313 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 42.55 μs 01:51:52,313 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 01:51:52,313 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=14 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, logIndex: 16, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 15, nextIndex: 16 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 15, lastAppliedIndex: 14 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 15 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 14 to 15 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 15 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@528ac905 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 14 for term 4 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 16, leaderLastIndex: 16, leaderSnapShotIndex: 14 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 16 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=15, prevLogTerm=4, leaderCommit=15, replicatedToAllIndex=14, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}]]] 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=15 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@41acc32 01:51:52,313 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 01:51:52,313 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, message: Envelope{sessionId=0, txSequence=19, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0}} 01:51:52,314 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 451 bytes, total slices to send: 1 01:51:52,314 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 01:51:52,314 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=15, prevLogTerm=4, leaderCommit=15, replicatedToAllIndex=14, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}]]] 01:51:52,314 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:51:52,314 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} with Envelope{sessionId=0, txSequence=17, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2}} 01:51:52,314 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 15, entries to be added from: 0 01:51:52,314 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77} 01:51:52,314 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 01:51:52,314 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866640450387, sessionId=0, txSequence=23} with Envelope{sessionId=0, txSequence=17, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2}} 01:51:52,314 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=DONE}: purge completed 01:51:52,314 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 01:51:52,314 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 15 01:51:52,314 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@96a341c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=DONE} 01:51:52,314 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=16, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:52,314 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 15, context.getLastApplied(): 14, lastIndex(): 16 01:51:52,314 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 15 01:51:52,314 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 14 to 15 01:51:52,314 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 15 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 01:51:52,314 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 01:51:52,314 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 14 for term 4 01:51:52,314 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 01:51:52,314 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=15 01:51:52,315 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 16, nextIndex: 17 01:51:52,315 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 16, lastAppliedIndex: 15 01:51:52,315 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 16 01:51:52,315 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 15 to 16 01:51:52,315 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=19, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0}} 01:51:52,315 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 16 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77} 01:51:52,315 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@dcff55 01:51:52,315 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=866642763344, sessionId=0, txSequence=25} with Envelope{sessionId=0, txSequence=19, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0}} 01:51:52,315 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0} 01:51:52,315 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:51:52,315 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:51:52,315 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 15 for term 4 01:51:52,315 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@96a341c aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} 01:51:52,315 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} allocated sequence 1 01:51:52,315 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 17, leaderLastIndex: 16, leaderSnapShotIndex: 15 01:51:52,315 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@9f7649f 01:51:52,315 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=16 01:51:52,315 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT}, enqueuedTicks=866645867631} 01:51:52,315 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, message: Envelope{sessionId=0, txSequence=1a, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT}} 01:51:52,315 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message size: 296 bytes, total slices to send: 1 01:51:52,315 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message does not need to be sliced - sending original message 01:51:52,315 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@af92f3d changed state from IDLE to TX_OPEN 01:51:52,315 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:51:52,315 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=OPEN} allocated sequence 0 01:51:52,315 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@382bafe 01:51:52,315 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY}, enqueuedTicks=866646266651} 01:51:52,315 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, message: Envelope{sessionId=0, txSequence=1b, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY}} 01:51:52,315 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message size: 455 bytes, total slices to send: 1 01:51:52,315 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message does not need to be sliced - sending original message 01:51:52,315 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@af92f3d changed state from TX_OPEN to IDLE 01:51:52,315 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@af92f3d readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 01:51:52,315 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=FLUSHED} allocated sequence 1 01:51:52,315 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@5dc6f1e 01:51:52,315 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE}, enqueuedTicks=866646524387} 01:51:52,315 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, message: Envelope{sessionId=0, txSequence=1c, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE}} 01:51:52,316 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message size: 296 bytes, total slices to send: 1 01:51:52,316 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message does not need to be sliced - sending original message 01:51:52,316 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 01:51:52,316 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 01:51:52,316 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, logIndex: 17, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@41acc32 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 ready 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}], operation=WRITE}}}}}}}]} 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 to ready 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 is already in state READY (READY) 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 initiating direct canCommit 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 validated 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 initiating direct preCommit 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:51:52,317 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} with Envelope{sessionId=0, txSequence=18, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2}} 01:51:52,317 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866642516848, sessionId=0, txSequence=24} with Envelope{sessionId=0, txSequence=18, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2}} 01:51:52,317 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=DONE}: purge completed 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 candidate ready 01:51:52,317 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@96a341c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=DONE} 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1: canCommit - candidate rootPath: / 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1: canCommit - messages: [] 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1: preCommit - successfulFromPrevious: [] 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 prepared 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 initiating direct commit 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, logIndex: 18, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 17, leaderLastIndex: 18, leaderSnapShotIndex: 15 01:51:52,317 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=1b, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0}} 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 17 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,317 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY}, enqueuedTicks=866646266651, sessionId=0, txSequence=27} with Envelope{sessionId=0, txSequence=1b, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0}} 01:51:52,317 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0} 01:51:52,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=16, prevLogTerm=4, leaderCommit=16, replicatedToAllIndex=15, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}], SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}]]] 01:51:52,318 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 submitted to persistence 01:51:52,318 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 01:51:52,318 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 16 01:51:52,318 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 16, context.getLastApplied(): 15, lastIndex(): 16 01:51:52,318 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 16 01:51:52,318 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 15 to 16 01:51:52,318 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 16 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77} 01:51:52,318 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 01:51:52,318 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 01:51:52,318 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 15 for term 4 01:51:52,318 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=16 01:51:52,318 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=16, prevLogTerm=4, leaderCommit=16, replicatedToAllIndex=15, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}], SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}]]] 01:51:52,318 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 01:51:52,318 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 16, entries to be added from: 0 01:51:52,318 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 01:51:52,318 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 01:51:52,318 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 01:51:52,318 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 01:51:52,318 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 01:51:52,318 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=18, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:52,319 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 01:51:52,319 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 01:51:52,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 17 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 18, nextIndex: 19 01:51:52,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 18, lastAppliedIndex: 16 01:51:52,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 17 01:51:52,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 16 to 17 01:51:52,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 17 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 01:51:52,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2284/0x00000008016d86d0@693d61df 01:51:52,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 01:51:52,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 18 01:51:52,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 17 to 18 01:51:52,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 18 data Simple{size=241} 01:51:52,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:51:52,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:51:52,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1: commit - successfulFromPrevious: [] 01:51:52,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 changing state from COMMIT_PENDING to COMMITTED 01:51:52,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}], operation=WRITE, modificationType=WRITE}}}}}}}]} 01:51:52,319 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:51:52,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 17 for term 4 01:51:52,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 19, leaderLastIndex: 18, leaderSnapShotIndex: 17 01:51:52,319 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 66.50 μs 01:51:52,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=18 01:51:52,319 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=1a, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1}} 01:51:52,319 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT}, enqueuedTicks=866645867631, sessionId=0, txSequence=26} with Envelope{sessionId=0, txSequence=1a, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1}} 01:51:52,319 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1} 01:51:52,319 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN}: initiating purge 01:51:52,319 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=DONE} allocated sequence 2 01:51:52,319 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@12a69169 01:51:52,319 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866650318951} 01:51:52,320 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=1c, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1}} 01:51:52,320 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=SIMPLE}, enqueuedTicks=866646524387, sessionId=0, txSequence=28} with Envelope{sessionId=0, txSequence=1c, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1}} 01:51:52,320 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=FLUSHED} directCommit completed 01:51:52,320 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 01:51:52,320 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 01:51:52,320 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 01:51:52,320 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, logIndex: 19, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:52,320 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=FLUSHED}: initiating purge 01:51:52,320 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=DONE} allocated sequence 2 01:51:52,320 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@684f8e47 01:51:52,320 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866650821353} 01:51:52,321 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 18 01:51:52,321 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 18, context.getLastApplied(): 16, lastIndex(): 18 01:51:52,321 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 17 01:51:52,321 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 16 to 17 01:51:52,321 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 17 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 01:51:52,321 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 01:51:52,321 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 18 01:51:52,321 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 17 to 18 01:51:52,321 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 18 data Simple{size=241} 01:51:52,321 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:51:52,321 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 01:51:52,321 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 19 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,321 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} allocated sequence 0 01:51:52,321 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@30a4942 01:51:52,321 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=18, prevLogTerm=4, leaderCommit=18, replicatedToAllIndex=17, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}]]] 01:51:52,321 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=866652095065} 01:51:52,321 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:51:52,321 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 01:51:52,321 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 01:51:52,321 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, logIndex: 20, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:52,321 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:51:52,321 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 01:51:52,321 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 17 for term 4 01:51:52,321 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=18 01:51:52,321 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 01:51:52,322 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 46.96 μs 01:51:52,322 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 01:51:52,322 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:51:52,322 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@cc25162 01:51:52,322 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 01:51:52,322 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=18, prevLogTerm=4, leaderCommit=18, replicatedToAllIndex=17, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}]]] 01:51:52,322 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, message: Envelope{sessionId=0, txSequence=1f, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0}} 01:51:52,322 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:51:52,322 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 18, entries to be added from: 0 01:51:52,322 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 01:51:52,322 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 01:51:52,322 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 01:51:52,322 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=19, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:52,322 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 471 bytes, total slices to send: 1 01:51:52,322 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 01:51:52,322 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 01:51:52,322 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 19, nextIndex: 20 01:51:52,322 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 19, lastAppliedIndex: 18 01:51:52,322 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 19 01:51:52,322 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 18 to 19 01:51:52,322 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 19 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 01:51:52,322 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@36cb9d2c 01:51:52,322 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 01:51:52,322 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 01:51:52,322 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 18 for term 4 01:51:52,322 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 20, leaderLastIndex: 20, leaderSnapShotIndex: 18 01:51:52,323 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 20 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,323 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=19, prevLogTerm=4, leaderCommit=19, replicatedToAllIndex=18, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}]]] 01:51:52,323 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=19 01:51:52,323 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=19, prevLogTerm=4, leaderCommit=19, replicatedToAllIndex=18, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}]]] 01:51:52,323 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:51:52,323 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 19, entries to be added from: 0 01:51:52,323 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77} 01:51:52,323 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 01:51:52,323 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 01:51:52,323 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 19 01:51:52,323 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=20, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:52,323 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 19, context.getLastApplied(): 18, lastIndex(): 20 01:51:52,324 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 19 01:51:52,324 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 18 to 19 01:51:52,324 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 19 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 01:51:52,324 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 01:51:52,324 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 18 for term 4 01:51:52,324 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=1f, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0}} 01:51:52,324 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 20, nextIndex: 21 01:51:52,324 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 20, lastAppliedIndex: 19 01:51:52,324 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=866652095065, sessionId=0, txSequence=31} with Envelope{sessionId=0, txSequence=1f, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0}} 01:51:52,324 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 20 01:51:52,324 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 19 to 20 01:51:52,324 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0} 01:51:52,324 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 20 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77} 01:51:52,324 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@7727e4b2 01:51:52,324 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} with Envelope{sessionId=0, txSequence=1d, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2}} 01:51:52,324 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:51:52,324 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866650318951, sessionId=0, txSequence=29} with Envelope{sessionId=0, txSequence=1d, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2}} 01:51:52,324 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=DONE}: purge completed 01:51:52,324 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@96a341c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=DONE} 01:51:52,324 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:51:52,324 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 19 for term 4 01:51:52,324 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 21, leaderLastIndex: 20, leaderSnapShotIndex: 19 01:51:52,324 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 01:51:52,324 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=20 01:51:52,324 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=19 01:51:52,324 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@96a341c aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} 01:51:52,324 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} allocated sequence 1 01:51:52,324 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@16044f58 01:51:52,324 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT}, enqueuedTicks=866655323815} 01:51:52,324 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, message: Envelope{sessionId=0, txSequence=20, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT}} 01:51:52,324 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message size: 296 bytes, total slices to send: 1 01:51:52,324 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message does not need to be sliced - sending original message 01:51:52,325 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} with Envelope{sessionId=0, txSequence=1e, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2}} 01:51:52,325 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866650821353, sessionId=0, txSequence=30} with Envelope{sessionId=0, txSequence=1e, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2}} 01:51:52,325 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=DONE}: purge completed 01:51:52,325 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@96a341c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=DONE} 01:51:52,325 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 01:51:52,325 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 20 01:51:52,325 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 20, context.getLastApplied(): 19, lastIndex(): 20 01:51:52,325 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 20 01:51:52,325 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 19 to 20 01:51:52,325 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 01:51:52,325 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 20 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77} 01:51:52,325 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 01:51:52,325 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 01:51:52,325 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, logIndex: 21, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 01:51:52,325 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 19 for term 4 01:51:52,325 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=20 01:51:52,325 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 21 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,325 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=20, prevLogTerm=4, leaderCommit=20, replicatedToAllIndex=19, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}]]] 01:51:52,325 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 01:51:52,326 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=20, prevLogTerm=4, leaderCommit=20, replicatedToAllIndex=19, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}]]] 01:51:52,326 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:51:52,326 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 20, entries to be added from: 0 01:51:52,326 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 01:51:52,326 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 01:51:52,326 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 01:51:52,326 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=21, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:52,326 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 01:51:52,326 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 21, nextIndex: 22 01:51:52,326 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 21, lastAppliedIndex: 20 01:51:52,326 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 21 01:51:52,326 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 20 to 21 01:51:52,326 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 21 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 01:51:52,326 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2284/0x00000008016d86d0@3701cd64 01:51:52,326 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 01:51:52,326 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 20 for term 4 01:51:52,326 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 22, leaderLastIndex: 21, leaderSnapShotIndex: 20 01:51:52,326 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=21 01:51:52,326 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=20, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1}} 01:51:52,326 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT}, enqueuedTicks=866655323815, sessionId=0, txSequence=32} with Envelope{sessionId=0, txSequence=20, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1}} 01:51:52,326 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1} 01:51:52,326 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN}: initiating purge 01:51:52,326 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=DONE} allocated sequence 2 01:51:52,326 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@2215a549 01:51:52,326 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:52,326 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866657423877} 01:51:52,326 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 21 01:51:52,326 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 21, context.getLastApplied(): 20, lastIndex(): 21 01:51:52,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 21 01:51:52,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 20 to 21 01:51:52,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 21 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 01:51:52,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 01:51:52,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 20 for term 4 01:51:52,327 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=21 01:51:52,327 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 01:51:52,327 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 01:51:52,327 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 01:51:52,327 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, logIndex: 22, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:52,327 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 01:51:52,437 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 22 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,437 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=21, prevLogTerm=4, leaderCommit=21, replicatedToAllIndex=20, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}]]] 01:51:52,438 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=21, prevLogTerm=4, leaderCommit=21, replicatedToAllIndex=20, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}]]] 01:51:52,438 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 01:51:52,438 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 21, entries to be added from: 0 01:51:52,438 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 01:51:52,438 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 01:51:52,438 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 01:51:52,438 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=22, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:52,438 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 01:51:52,439 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 22, nextIndex: 23 01:51:52,439 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 22, lastAppliedIndex: 21 01:51:52,439 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 22 01:51:52,439 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 21 to 22 01:51:52,439 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 22 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 01:51:52,439 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@13fda665 01:51:52,439 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 01:51:52,439 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 01:51:52,439 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 21 for term 4 01:51:52,439 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 23, leaderLastIndex: 22, leaderSnapShotIndex: 21 01:51:52,439 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=22 01:51:52,440 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]} with Envelope{sessionId=0, txSequence=21, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2}} 01:51:52,440 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#669163223]}, enqueuedTicks=866657423877, sessionId=0, txSequence=33} with Envelope{sessionId=0, txSequence=21, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2}} 01:51:52,440 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=DONE}: purge completed 01:51:52,440 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@96a341c purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=DONE} 01:51:52,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 22 01:51:52,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 22, context.getLastApplied(): 21, lastIndex(): 22 01:51:52,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 22 01:51:52,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 21 to 22 01:51:52,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 22 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 01:51:52,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 01:51:52,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 21 for term 4 01:51:52,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=22 01:51:52,825 PM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1808634169] 01:51:52,825 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: findLocalShard : cars 01:51:52,826 PM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491] 01:51:52,826 PM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.TestShard$RequestFrontendMetadata to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491] 01:51:52,827 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 01:51:52,827 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} allocated sequence 0 01:51:52,827 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@4ab5c815 01:51:52,827 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:52,827 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=867157753172} 01:51:52,828 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1 01:51:52,828 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:51:52,828 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@cc25162 01:51:52,828 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, message: Envelope{sessionId=0, txSequence=22, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0}} 01:51:52,828 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message size: 460 bytes, total slices to send: 1 01:51:52,828 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-674: Message does not need to be sliced - sending original message 01:51:52,830 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=22, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0}} 01:51:52,830 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#669163223], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=867157753172, sessionId=0, txSequence=34} with Envelope{sessionId=0, txSequence=22, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0}} 01:51:52,830 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0} 01:51:52,830 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@96a341c aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} 01:51:52,830 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} allocated sequence 1 01:51:52,830 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@15cf0c37 01:51:52,830 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:52,830 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT}, enqueuedTicks=867161462694} 01:51:52,831 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, message: Envelope{sessionId=0, txSequence=23, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#669163223], modifications=0, protocol=ABORT}} 01:51:52,831 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message size: 296 bytes, total slices to send: 1 01:51:52,831 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-675: Message does not need to be sliced - sending original message 01:51:52,831 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:51:52,831 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Issuing gracefulStop to shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:52,831 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession - waiting on 1 shards 01:51:52,831 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onShutDown 01:51:52,831 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Initiating leader transfer 01:51:52,831 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=null, leaderPayloadVersion=13] 01:51:52,831 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5ddfd2dc 01:51:52,831 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:52,831 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:52,831 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 01:51:52,831 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5ddfd2dc 01:51:52,831 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Attempting to transfer leadership 01:51:52,832 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 23, leaderLastIndex: 22, leaderSnapShotIndex: 21 01:51:52,832 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: not currently active leader, rejecting request Envelope{sessionId=0, txSequence=23, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#669163223], modifications=0, protocol=ABORT}}. isLeader: true, isLeaderActive: false,isLeadershipTransferInProgress: true, paused: true 01:51:52,832 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: request Envelope{sessionId=0, txSequence=23, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#669163223], modifications=0, protocol=ABORT}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:51:52,832 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession] 01:51:52,832 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=null, leaderPayloadVersion=13] 01:51:52,832 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:52,832 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:52,832 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@5c091f79, splicing queue 01:51:52,832 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:52,832 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50487f8c queue spliced 1 messages 01:51:52,832 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:52,833 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: refreshing backend for shard 1 01:51:52,833 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:52,833 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:52,833 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:52,833 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-1162099069] 01:51:52,833 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:52,833 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:51:52,836 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, matchIndex: 22, lastIndex: 22, isVoting: true 01:51:52,836 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Follower's log matches - sending ElectionTimeout 01:51:52,836 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 23, leaderLastIndex: 22, leaderSnapShotIndex: 21 01:51:52,836 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Leader transfer complete 01:51:53,40 PM [cluster-test-shard-dispatcher-26] [WARN] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=22, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 202, lastApplied : 22, commitIndex : 22 01:51:53,58 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:51:53,247 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All shards shutdown - sending PoisonPill to self 01:51:53,247 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#1829786491]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:51:53,247 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:53,248 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader)_slicer-id-677: Closing 01:51:53,248 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborting 0 pending queued transactions 01:51:53,248 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 01:51:53,248 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 01:51:53,248 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 01:51:53,248 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: shutting down 01:51:53,248 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 01:51:53,248 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 01:51:53,249 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1808634169] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1808634169] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:53,249 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1808634169] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1808634169] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:53,249 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#-1501939593] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1501939593] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:53,249 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1808634169] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1808634169] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:53,249 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#-1501939593] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1501939593] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:53,249 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:53,249 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:53,253 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:53,253 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:53,253 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:53,253 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:53,255 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:51:53,267 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@57cb640d aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} 01:51:53,267 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:51:53,267 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2063144426], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@4adf44da 01:51:53,267 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} 01:51:53,267 PM [Cleaner-1] [DEBUG] RemoteProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} has a successor, which should receive seal through a separate request 01:51:53,267 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:51:53,267 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2063144426], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1240/0x0000000801303148@18446c08 01:51:53,267 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:53,267 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1939334546], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:53,267 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2c5607eb aborted transaction LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 01:51:53,267 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2063144426]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@4fc97505 01:51:53,267 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@34966feb transmitting entry ConnectionEntry{request=AbortLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2063144426]}, enqueuedTicks=867598457220} 01:51:53,268 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0} 01:51:53,268 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3f89f63c aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} 01:51:53,268 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} allocated sequence 1 01:51:53,268 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@71269c95 01:51:53,268 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#957377664], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:53,268 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#957377664], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:53,268 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1972fd95 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} 01:51:53,268 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} allocated sequence 1 01:51:53,268 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1472770649], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@416fefff 01:51:53,268 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1228987935], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:53,268 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1228987935], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:51:53,268 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0} 01:51:53,268 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5ccfc0db aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} 01:51:53,268 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:51:53,268 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1775345402], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@7af0eff6 01:51:53,268 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} 01:51:53,268 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SEALED} allocated sequence 2 01:51:53,268 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1775345402], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1240/0x0000000801303148@7e15ccf0 01:51:53,268 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:53,268 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1561463123], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:53,278 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 01:51:53,278 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 01:51:53,278 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Closing 01:51:53,278 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborting 0 pending queued transactions 01:51:53,279 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 01:51:53,279 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:53,279 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:54,266 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [7215635008473551815] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:54,296 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-3232233343804304326] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:54,300 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:54,300 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:54,301 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:54.302] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:54,310 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:54,310 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:54,312 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:54.313] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:54,336 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:51:54,340 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [6380823976918517814] 01:51:54,340 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:51:54,341 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:51:54,341 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:51:54,342 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:54,342 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:54,350 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:51:54,350 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:51:54,350 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:51:54,353 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:54,360 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-7000250799160867469] 01:51:54,360 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:51:54,360 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:54,360 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:51:54,360 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:54,360 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:54,375 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:51:54,383 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-7666628846392480242] 01:51:54,383 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:51:54,384 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:54,384 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:51:54,393 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:54,393 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:54,394 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:54,395 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:54,396 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 01:51:54,407 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1498272974] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:54,407 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:51:54,414 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 01:51:54,414 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:51:54,415 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 01:51:54,415 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5d1ad45c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4d1fd508 01:51:54,415 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@66d9f86d 01:51:54,415 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:54,415 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:54,415 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:54,415 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1710165114415, None)) 01:51:54,415 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165114414,0,0)) 01:51:54,416 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4d1fd508 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1b4a3b40 01:51:54,416 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 01:51:54,416 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:54,417 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1a9925b0 01:51:54,417 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:54,417 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:54,418 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:54,420 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 01:51:54,425 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:51:54,427 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:51:54,427 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:54,427 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:54,427 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821] 01:51:54,427 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 01:51:54,427 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:54,428 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#1423348479 created and ready for shard:member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:54,428 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#1423348479] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:54,428 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-1:snapshot-read#617017646] 01:51:54,428 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-1:snapshot-read#617017646] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:54,428 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 01:51:54,429 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:54,429 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:54,429 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:54,429 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:54,431 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:51:54,431 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:51:54,431 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 01:51:54,431 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 01:51:54,431 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 01:51:54,431 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 01:51:54,433 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 01:51:54,434 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:51:54,434 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-730636097] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:54,436 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 01:51:54,436 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:51:54,436 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7eadb84f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2b4c001f 01:51:54,436 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@35e8bb5c 01:51:54,436 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 01:51:54,436 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1710165114436, None)) 01:51:54,436 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:54,436 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165114435,0,0)) 01:51:54,436 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:54,436 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:54,436 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3e2b61a6 01:51:54,436 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2b4c001f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@202b63d5 01:51:54,436 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 01:51:54,436 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:54,436 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:54,437 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:54,437 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:54,437 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop#2005335308] 01:51:54,437 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 01:51:54,437 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop#2005335308] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:54,437 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 01:51:54,437 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-2086105701 created and ready for shard:member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:54,437 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-2:snapshot-read#-857924626] 01:51:54,437 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-2086105701] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:54,437 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-2:snapshot-read#-857924626] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:54,438 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:54,438 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:54,438 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:51:54,438 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:51:54,438 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:54,438 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 01:51:54,438 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 01:51:54,438 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 01:51:54,438 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 01:51:54,439 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 01:51:54,487 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1498272974] 01:51:54,487 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: findLocalShard : cars 01:51:54,487 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821] 01:51:54,690 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:54,690 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Received ElectionTimeout - switching to Candidate 01:51:54,690 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:51:54,690 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=1, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:54,690 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 1 01:51:54,690 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:54,690 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 01:51:54,690 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:51:54,690 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:51:54,690 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Follower to Candidate 01:51:54,690 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Follower to Candidate 01:51:54,960 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 01:51:54,960 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=2, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:54,961 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 2 01:51:54,981 PM [cluster-test-shard-dispatcher-24] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:55,280 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 01:51:55,280 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=3, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,280 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 3 01:51:55,362 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:55,362 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,362 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,362 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:55,362 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,362 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,362 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:51:55,362 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,362 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:51:55,362 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,396 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:55,396 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,396 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,397 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:55,397 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,397 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,397 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,413 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:51:55,561 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 01:51:55,561 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,561 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 4 01:51:55,561 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,561 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,563 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:55,563 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:55,563 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 01:51:55,563 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:55,563 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:55,563 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=null 01:51:55,563 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:55,563 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,563 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:55,564 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:55,564 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:55,564 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:51:55,564 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:55,564 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:55,564 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:55,564 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 01:51:55,564 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: became leader with frontend state for [] 01:51:55,564 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Candidate to Leader 01:51:55,564 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 01:51:55,564 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:55,564 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:55,564 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Candidate to Leader 01:51:55,564 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 01:51:55,564 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:55,565 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 01:51:55,565 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: getLeaderAddress leaderId = member-1-shard-cars-testRaftCallbackDuringLeadershipDrop peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,565 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 01:51:55,565 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:55,565 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:55,565 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 01:51:55,565 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done false 01:51:55,565 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 01:51:55,565 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:55,603 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5dde1c99: creating a new transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:51:55,603 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} 01:51:55,603 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:55,603 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:55,603 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1498272974] 01:51:55,603 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:55,603 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} shard 1 01:51:55,604 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:51:55,604 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2fddd2cc] 01:51:55,604 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:55,604 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5dde1c99 readied transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 01:51:55,604 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1957841268], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@7a7877df 01:51:55,604 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2fddd2cc] 01:51:55,604 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:51:55,604 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@7233696, attempting to connect 01:51:55,604 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} is not yet known 01:51:55,604 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, nanosAgo=15540, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:51:55,605 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:51:55,605 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:51:55,605 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:55,605 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@35ccdce9 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@25fbc77b 01:51:55,605 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:51:55,605 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@25fbc77b 01:51:55,605 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:51:55,605 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=OPEN} 01:51:55,605 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1957841268], modifications=1, protocol=SIMPLE} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=OPEN} 01:51:55,605 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1957841268], modifications=1, protocol=SIMPLE} 01:51:55,605 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SEALED} allocated sequence 0 01:51:55,605 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1957841268], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@7a7877df 01:51:55,606 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:51:55,606 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58702d12 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1957841268], coordinated=false}, enqueuedTicks=869934744610} 01:51:55,606 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5dde1c99 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:55,606 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} 01:51:55,606 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@23cbe978 01:51:55,606 PM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: allocating new ready transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:51:55,606 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4801f6fd superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@5157be2c, splicing queue 01:51:55,606 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4801f6fd queue spliced 0 messages 01:51:55,606 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 initiating direct canCommit 01:51:55,606 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Validating transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:51:55,606 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 validated 01:51:55,606 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:55,606 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 1.234 ms 01:51:55,606 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 initiating direct preCommit 01:51:55,606 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Preparing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:51:55,606 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 candidate ready 01:51:55,606 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1: canCommit - candidate rootPath: / 01:51:55,606 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1: canCommit - messages: [] 01:51:55,606 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 01:51:55,606 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 prepared 01:51:55,606 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:55,606 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 initiating direct commit 01:51:55,606 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Starting commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:51:55,607 PM [cluster-test-shard-dispatcher-33] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 174 01:51:55,607 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:51:55,607 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:51:55,607 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:55,607 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:55,607 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:55,607 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,607 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 01:51:55,607 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 submitted to persistence 01:51:55,608 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:55,608 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:51:55,609 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 01:51:55,609 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 1 01:51:55,609 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:55,609 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=174} 01:51:55,609 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:51:55,609 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 1 01:51:55,609 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:55,609 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:51:55,609 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done true 01:51:55,609 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 0, nextIndex: 1 01:51:55,609 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:51:55,609 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Setting last applied to 0 01:51:55,609 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 01:51:55,609 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data Simple{size=174} 01:51:55,609 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Resuming commit of transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:51:55,610 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendMetadata - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4caaf3f}, purged=MutableUnsignedLongSet{size=0}} 01:51:55,610 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 01:51:55,610 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:51:55,610 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1: commit - successfulFromPrevious: [] 01:51:55,610 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:51:55,610 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1957841268], coordinated=false} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0}} 01:51:55,610 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-1847157020] with name member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 01:51:55,610 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1957841268], coordinated=false}, enqueuedTicks=869934744610, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0}} 01:51:55,610 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} directCommit completed 01:51:55,610 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-1847157020] 01:51:55,610 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 01:51:55,610 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:51:55,610 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:55,610 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1957841268]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@81611a9 01:51:55,610 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} 01:51:55,610 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:51:55,610 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1957841268]} 01:51:55,610 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SEALED}: initiating purge 01:51:55,610 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:51:55,610 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=DONE} allocated sequence 1 01:51:55,610 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1957841268]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@746ab9fa 01:51:55,610 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:55,610 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=0 01:51:55,610 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58702d12 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1957841268]}, enqueuedTicks=869940913352} 01:51:55,610 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 110.3 μs 01:51:55,610 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: purging transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:51:55,610 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-1847157020] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:55,610 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 01:51:55,610 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 01:51:55,611 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:55,611 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 01:51:55,611 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:55,612 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 01:51:55,614 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#259762366] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:55,615 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:51:55,615 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 0 01:51:55,615 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:51:55,615 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 0 01:51:55,615 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 01:51:55,615 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data Simple{size=174} 01:51:55,615 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:51:55,615 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@2bbcf38d}, purged=MutableUnsignedLongSet{size=0}} 01:51:55,615 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 01:51:55,615 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:51:55,615 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-713509570] with name member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 01:51:55,616 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=0 01:51:55,617 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,617 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}]]] 01:51:55,617 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 01:51:55,617 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 01:51:55,618 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:51:55,618 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,618 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@c24f070 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@bd147ac 01:51:55,618 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@40ceb0f9 01:51:55,618 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}]]] 01:51:55,618 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 1 01:51:55,618 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:55,618 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:51:55,618 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,618 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:55,618 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,618 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:55,618 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,618 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1710165115618, None)) 01:51:55,618 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 01:51:55,618 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 01:51:55,618 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 2 01:51:55,618 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:55,618 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165115617,0,0)) 01:51:55,618 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@bd147ac to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@14d44797 01:51:55,619 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 01:51:55,619 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6a986f48 01:51:55,619 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 1, nextIndex: 2 01:51:55,619 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-713509570] 01:51:55,619 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:51:55,619 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Setting last applied to 1 01:51:55,619 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 01:51:55,619 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 01:51:55,619 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: replication of member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@207caa83 01:51:55,619 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:51:55,619 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:55,619 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,619 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#259762366] 01:51:55,619 PM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: finished purging transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:51:55,619 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 01:51:55,619 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Purged transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:51:55,619 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 79.45 μs 01:51:55,619 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,619 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:51:55,619 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,619 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#371673895] 01:51:55,619 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,619 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,619 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-713509570] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:55,619 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 01:51:55,619 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#371673895] 01:51:55,619 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,619 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,619 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#371673895] 01:51:55,619 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-730636097] 01:51:55,619 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5dde1c99: creating a new transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:51:55,619 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:51:55,619 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 01:51:55,620 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1498272974] 01:51:55,620 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1957841268]} with Envelope{sessionId=0, txSequence=1, message=TransactionPurgeResponse{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1}} 01:51:55,620 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1957841268]}, enqueuedTicks=869940913352, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPurgeResponse{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1}} 01:51:55,620 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=DONE}: purge completed 01:51:55,620 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@25fbc77b purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=DONE} 01:51:55,620 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:51:55,620 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@35ccdce9 purged transaction RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} 01:51:55,620 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 01:51:55,620 PM [pool-35-thread-1] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5dde1c99 readied transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 01:51:55,620 PM [pool-35-thread-1] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=FLUSHED} allocated sequence 0 01:51:55,620 PM [pool-35-thread-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1957841268], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@4862287c 01:51:55,620 PM [pool-35-thread-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:55,620 PM [pool-35-thread-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58702d12 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1957841268], coordinated=false}, enqueuedTicks=869951106765} 01:51:55,620 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#371673895] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:55,620 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#1047477500 created and ready for shard:member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,620 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#1047477500] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:55,620 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 01:51:55,620 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 1 01:51:55,620 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:51:55,620 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 1 01:51:55,620 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 01:51:55,620 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 01:51:55,620 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-3:snapshot-read#-420269687] 01:51:55,621 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 01:51:55,621 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Purged transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:51:55,621 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: allocating new ready transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:51:55,621 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-3:snapshot-read#-420269687] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 initiating direct canCommit 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Validating transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 validated 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 initiating direct preCommit 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Preparing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 candidate ready 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1: canCommit - candidate rootPath: / 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1: canCommit - messages: [] 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 prepared 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 initiating direct commit 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Starting commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 72 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}]]] 01:51:55,621 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 submitted to persistence 01:51:55,622 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,622 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:55,623 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:55,623 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:51:55,623 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:51:55,623 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 01:51:55,623 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 01:51:55,623 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=null 01:51:55,623 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 01:51:55,623 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: getLeaderAddress leaderId = member-1-shard-cars-testRaftCallbackDuringLeadershipDrop peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,623 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 01:51:55,623 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 01:51:55,623 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 01:51:55,623 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:51:55,623 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,623 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}]]] 01:51:55,623 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,623 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:55,624 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done false 01:51:55,624 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 01:51:55,624 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 01:51:55,624 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:55,624 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:55,624 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 01:51:55,624 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 01:51:55,627 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 01:51:55,721 PM [awaitility-thread] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821] 01:51:55,722 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821] 01:51:55,722 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:55,722 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Cluster state: [] 01:51:55,722 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:55,723 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Term 5 in "RequestVote [term=5, candidateId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1]" message is greater than leader's term 4 - switching to Follower 01:51:55,723 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=5, votedFor=null 01:51:55,723 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader) :- Switching from behavior Leader to Follower, election term: 5 01:51:55,723 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader)_slicer-id-682: Closing 01:51:55,723 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:51:55,723 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:51:55,723 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: removing frontend state for [member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop] 01:51:55,723 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 01:51:55,723 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Leader to Follower 01:51:55,723 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=-1] 01:51:55,723 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:55,723 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Leader to Follower 01:51:55,723 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:55,723 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58702d12 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@24229441, splicing queue 01:51:55,723 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:51:55,723 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58702d12 queue spliced 1 messages 01:51:55,723 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:55,723 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: refreshing backend for shard 1 01:51:55,723 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:51:55,723 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:55,723 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:55,723 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1498272974] 01:51:55,724 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:55,724 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 01:51:56,41 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:56,41 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Received ElectionTimeout - switching to Candidate 01:51:56,41 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:51:56,41 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:56,41 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 6 01:51:56,41 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] to peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:56,41 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] to peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:56,41 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower) :- Switching from behavior Follower to Candidate, election term: 6 01:51:56,41 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 01:51:56,41 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 01:51:56,41 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 01:51:56,41 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Follower to Candidate 01:51:56,41 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Follower to Candidate 01:51:56,41 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized - notifying 1 OnShardInitialized callbacks 01:51:56,42 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:56,42 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 01:51:56,42 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 01:51:56,42 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 6 in "RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 4 - updating term 01:51:56,42 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:56,42 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=null 01:51:56,42 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 1, lastTerm: 4 01:51:56,42 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:56,42 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 01:51:56,42 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 01:51:56,42 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 01:51:56,42 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 6 in "RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 4 - updating term 01:51:56,42 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=null 01:51:56,43 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:56,43 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:56,43 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 01:51:56,43 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=true], my term is 6 01:51:56,43 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=true], current voteCount: 1 01:51:56,43 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): LastApplied index 1 is behind last index 2 - switching to PreLeader 01:51:56,43 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:51:56,43 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:51:56,43 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:51:56,44 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 6 01:51:56,44 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 01:51:56,44 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: became leader with frontend state for [member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop] 01:51:56,44 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Candidate to PreLeader 01:51:56,44 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] 01:51:56,44 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 01:51:56,44 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] 01:51:56,44 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized - notifying 1 OnShardInitialized callbacks 01:51:56,44 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Replicate message: identifier: null, logIndex: 3, payload: class org.opendaylight.controller.cluster.raft.persisted.NoopPayload, isSendImmediate: true 01:51:56,44 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:56,44 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:51:56,44 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Candidate to PreLeader 01:51:56,44 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:56,44 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized - notifying 1 OnShardInitialized callbacks 01:51:56,44 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:51:56,44 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): The followers log is empty and the senders prevLogIndex is 0 01:51:56,44 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:56,44 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] 01:51:56,44 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done true 01:51:56,45 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 1, nextIndex: 2 01:51:56,45 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:51:56,45 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:56,45 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e]]] 01:51:56,45 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): The followers log is empty and the senders prevLogIndex is 0 01:51:56,45 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:56,45 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 01:51:56,45 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 01:51:56,45 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 01:51:56,45 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 3, leaderSnapShotIndex: -1 01:51:56,45 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:56,45 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e]]] 01:51:56,46 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 01:51:56,46 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 01:51:56,46 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:56,46 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e]]] 01:51:56,46 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 2 01:51:56,46 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:51:56,46 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=72} 01:51:56,46 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 01:51:56,46 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e 01:51:56,46 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] 01:51:56,46 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 01:51:56,46 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:56,46 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e]]] 01:51:56,46 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 4 01:51:56,46 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:56,46 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=174} 01:51:56,46 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 01:51:56,46 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:51:56,46 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] 01:51:56,46 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 01:51:56,46 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=72} 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 1 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 3 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 0 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data Simple{size=174} 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@13ca3077}, purged=MutableUnsignedLongSet{size=0}} 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#1997496167] with name member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 1 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 01:51:56,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Purged transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 01:51:56,48 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#1997496167] 01:51:56,48 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:51:56,48 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 116.5 μs 01:51:56,48 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#1997496167] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:56,48 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 01:51:56,48 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 01:51:56,48 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 01:51:56,48 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] 01:51:56,48 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 3, nextIndex: 4 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 4 does not match the current term 6 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Setting last applied to 2 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, size=72} 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Resuming commit of transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1: commit - successfulFromPrevious: [] 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Setting last applied to 3 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 01:51:56,49 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} not found in inflight queue, checking pending queue 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 1 for term 6 01:51:56,49 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1957841268], coordinated=false} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} 01:51:56,49 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1957841268], coordinated=false}, enqueuedTicks=869951106765, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 1 01:51:56,49 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=FLUSHED} directCommit completed 01:51:56,49 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=FLUSHED}: initiating purge 01:51:56,49 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} allocated sequence 1 01:51:56,49 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1957841268]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@70587d66 01:51:56,49 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:56,49 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:56,49 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e]]] 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 3, nextIndex: 4 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 2 for term 6 01:51:56,49 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 13.53 μs 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:51:56,49 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Received ApplyState [identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}]] - lastApplied: 3, lastIndex: 3 01:51:56,50 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:51:56,50 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:51:56,50 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:51:56,50 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 6 01:51:56,50 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader)_slicer-id-685: Closing 01:51:56,50 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 01:51:56,50 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from PreLeader to Leader 01:51:56,50 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 01:51:56,50 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from PreLeader to Leader 01:51:56,50 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is ready - notifying 1 OnShardInitialized callbacks 01:51:56,50 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2fddd2cc] 01:51:56,50 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 01:51:56,50 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2fddd2cc] 01:51:56,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 3 01:51:56,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 01:51:56,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 2 01:51:56,50 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@17c3ca0, attempting to connect 01:51:56,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 01:51:56,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{size=72} 01:51:56,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:51:56,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 01:51:56,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:51:56,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 3 01:51:56,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 01:51:56,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e 01:51:56,50 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:51:56,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 1 for term 6 01:51:56,50 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:51:56,50 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@25fbc77b instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@40687fb0 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} previous state DONE 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} replaying messages to old proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@40687fb0 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=OPEN} 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 67.40 μs 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Replaying queued request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1957841268]} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=OPEN} 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1957841268]} 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=OPEN}: initiating purge 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} allocated sequence 0 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1957841268]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@4796174e 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7416e197 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1957841268]}, enqueuedTicks=870380277038} 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5dde1c99 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@7780855a 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@bd647d4 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@7c0d0d87, splicing queue 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@bd647d4 queue spliced 0 messages 01:51:56,51 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 414.2 μs 01:51:56,51 PM [cluster-test-shard-dispatcher-11] [WARN] AbstractFrontendHistory - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 not tracked in MutableUnsignedLongSet{span=[0..0], size=1}, but not present in active transactions 01:51:56,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e]]] 01:51:56,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 2 01:51:56,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] is present: existingEntryTerm: 4 01:51:56,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] is present: existingEntryTerm: 6 01:51:56,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 3, entries to be added from: 2 01:51:56,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 01:51:56,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 3 01:51:56,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:56,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 01:51:56,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 2 01:51:56,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 01:51:56,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{size=72} 01:51:56,51 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:51:56,52 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 01:51:56,52 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 01:51:56,52 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 3 01:51:56,52 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 01:51:56,52 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e 01:51:56,52 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 1 for term 6 01:51:56,52 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 01:51:56,52 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 01:51:56,52 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 20.68 μs 01:51:56,52 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1957841268]} with Envelope{sessionId=1, txSequence=0, message=TransactionPurgeResponse{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} 01:51:56,52 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1957841268]}, enqueuedTicks=870380277038, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionPurgeResponse{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} 01:51:56,52 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE}: purge completed 01:51:56,52 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@40687fb0 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} 01:51:56,52 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR}: purge completed 01:51:56,52 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@25fbc77b purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} 01:51:56,125 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:51:56,125 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:56,125 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 01:51:56,125 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 01:51:56,125 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:56,125 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 01:51:56,125 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 01:51:56,126 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 01:51:56,126 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:51:56,126 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:51:56,126 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:51:56,126 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 01:51:56,126 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:51:56,126 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#259762366] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#259762366] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:56,126 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#259762366] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#259762366] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:56,127 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:56,127 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 01:51:56,127 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 01:51:56,127 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Initiating leader transfer 01:51:56,127 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=13] 01:51:56,127 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:56,127 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@19d45180 01:51:56,127 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:56,127 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@19d45180 01:51:56,127 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Attempting to transfer leadership 01:51:56,127 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7416e197 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@3fd451c8, splicing queue 01:51:56,127 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7416e197 queue spliced 0 messages 01:51:56,127 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:51:56,127 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:51:56,127 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:51:56,127 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: refreshing backend for shard 1 01:51:56,127 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:51:56,127 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:56,127 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:56,127 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1498272974] 01:51:56,128 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:56,128 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 01:51:56,128 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop] 01:51:56,128 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 2 for term 6 01:51:56,128 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=13] 01:51:56,128 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:56,128 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:56,129 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, matchIndex: 3, lastIndex: 3, isVoting: true 01:51:56,129 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Follower's log matches - sending ElectionTimeout 01:51:56,129 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:51:56,129 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:51:56,129 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Leader transfer complete 01:51:56,381 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:51:56,540 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 01:51:56,540 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received ShardNotInitializedTimeout message for shard cars 01:51:56,540 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Returning NoShardLeaderException for shard cars 01:51:56,540 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#621648821]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:51:56,540 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:51:56,540 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - Stopping Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:56,540 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader)_slicer-id-686: Closing 01:51:56,541 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 01:51:56,541 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:56,541 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:51:56,541 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:56,541 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:51:56,541 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 01:51:56,541 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 01:51:56,541 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:51:56,541 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:51:56,541 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#-1957841268] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1957841268] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:56,541 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1498272974] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1498272974] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:56,541 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#-1957841268] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1957841268] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:56,541 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:51:56,542 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:56,542 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:56,549 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:56,549 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:56,549 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:56,549 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:56,551 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:51:56,561 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] MessageAssembler - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 01:51:56,562 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 01:51:56,562 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 01:51:56,562 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 01:51:56,562 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 01:51:56,563 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:56,563 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:57,563 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [6380823976918517814] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:57,584 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:57,584 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:57,584 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-7000250799160867469] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:51:57,587 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:57.589] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:57,593 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:57,593 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:51:57,595 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:51:57.596] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:57,608 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:51:57,614 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-7818456595557589733] 01:51:57,614 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:51:57,616 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:51:57,616 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:51:57,616 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:57,616 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:57,633 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:51:57,633 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:51:57,633 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:51:57,634 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:51:57,643 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [277931340935473324] 01:51:57,643 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:51:57,644 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:57,644 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:51:57,647 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:57,647 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:57,660 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:51:57,665 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [2221221624707611402] 01:51:57,665 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:51:57,666 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:51:57,666 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:51:57,673 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:51:57,673 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:51:57,674 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:57,675 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:57,676 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithSingleShard 01:51:57,682 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1939924251] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:57,683 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithSingleShard 01:51:57,686 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithSingleShard 01:51:57,686 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Creating local shard: member-1-shard-cars-testTransactionChainWithSingleShard 01:51:57,686 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:57,686 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:51:57,687 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: no initial generation override, starting from 0 01:51:57,687 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:51:57,687 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@463995ce to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@76afbfd8 01:51:57,687 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@33ad2f69 01:51:57,687 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionChainWithSingleShard, 0, 1710165117687, None)) 01:51:57,687 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165117686,0,0)) 01:51:57,688 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@76afbfd8 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@28696075 01:51:57,688 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} started 01:51:57,688 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:57,689 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:51:57,691 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithSingleShard 01:51:57,691 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@d0f0f30 01:51:57,691 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:57,691 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionChainWithSingleShard 01:51:57,691 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:51:57,691 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:51:57,691 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: updated peer member-1-shard-cars-testTransactionChainWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard on shard member-1-shard-cars-testTransactionChainWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518] 01:51:57,692 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionChainWithSingleShard, persistent : true 01:51:57,692 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:57,692 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:51:57,692 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-notifier#-1195335802 created and ready for shard:member-1-shard-cars-testTransactionChainWithSingleShard 01:51:57,692 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-notifier#-1195335802] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:57,692 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-member-1:snapshot-read#-488611741] 01:51:57,692 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-member-1:snapshot-read#-488611741] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:57,696 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionChainWithSingleShard with journal batch size 1 01:51:57,699 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:57,700 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithSingleShard 01:51:57,700 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#2060711106] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:57,701 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:57,701 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:57,701 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Initializing shard [cars] 01:51:57,701 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionChainWithSingleShard is initialized 01:51:57,701 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:51:57,701 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithSingleShard is no longer the leader 01:51:57,701 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from null to Follower 01:51:57,701 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard 01:51:57,701 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from null to Follower 01:51:57,701 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:51:57,703 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:51:57,704 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: no initial generation override, starting from 0 01:51:57,704 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:51:57,704 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@48f0470e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1ff70a9f 01:51:57,705 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1bc89db9 01:51:57,705 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithSingleShard 01:51:57,705 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Creating local shard: member-2-shard-cars-testTransactionChainWithSingleShard 01:51:57,705 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:51:57,705 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:51:57,705 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionChainWithSingleShard, 0, 1710165117705, None)) 01:51:57,705 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165117704,0,0)) 01:51:57,705 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1ff70a9f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4f3dfce2 01:51:57,705 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} started 01:51:57,706 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@578ad65b 01:51:57,706 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:51:57,706 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionChainWithSingleShard 01:51:57,706 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:51:57,706 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:51:57,706 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: updated peer member-1-shard-cars-testTransactionChainWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard on shard member-2-shard-cars-testTransactionChainWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard#-737658659] 01:51:57,706 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionChainWithSingleShard, persistent : true 01:51:57,706 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionChainWithSingleShard with journal batch size 1 01:51:57,707 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard#-737658659] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:57,707 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-notifier#140619564 created and ready for shard:member-2-shard-cars-testTransactionChainWithSingleShard 01:51:57,707 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/shard-member-2:snapshot-read#478158917] 01:51:57,707 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-notifier#140619564] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:57,707 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/shard-member-2:snapshot-read#478158917] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:57,707 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:51:57,707 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:51:57,707 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:51:57,707 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionChainWithSingleShard is no longer the leader 01:51:57,707 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Initializing shard [cars] 01:51:57,707 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionChainWithSingleShard is initialized 01:51:57,707 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionChainWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:51:57,707 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithSingleShard , received role change from null to Follower 01:51:57,707 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard 01:51:57,707 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-2-shard-cars-testTransactionChainWithSingleShard from null to Follower 01:51:57,756 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1939924251] 01:51:57,756 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: findLocalShard : cars 01:51:57,756 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518] 01:51:58,17 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:51:58,17 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 01:51:58,17 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionChainWithSingleShard] 01:51:58,17 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 01:51:58,17 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 1 01:51:58,17 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:51:58,17 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Closing 01:51:58,17 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:51:58,17 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithSingleShard is no longer the leader 01:51:58,17 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from Follower to Candidate 01:51:58,17 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from Follower to Candidate 01:51:58,303 PM [cluster-test-shard-dispatcher-25] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:51:58,327 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 01:51:58,327 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 01:51:58,327 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 2 01:51:58,607 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 01:51:58,607 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 01:51:58,607 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 3 01:51:58,627 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:51:58,627 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:51:58,628 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:58,628 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard 01:51:58,628 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:51:58,628 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:58,628 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:51:58,628 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:51:58,628 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionChainWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:51:58,636 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:51:58,636 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard 01:51:58,636 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:51:58,636 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:51:58,636 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:51:58,636 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 01:51:58,691 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:51:58,837 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 01:51:58,837 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 01:51:58,837 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 4 01:51:58,837 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionChainWithSingleShard 01:51:58,838 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:51:58,838 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Cluster state: [] 01:51:58,838 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:51:58,838 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:51:58,838 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=null 01:51:58,839 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:51:58,839 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 01:51:58,839 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:51:58,840 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:51:58,840 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:51:58,840 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionChainWithSingleShard] 01:51:58,840 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:58,840 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:51:58,840 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:51:58,840 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: became leader with frontend state for [] 01:51:58,840 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from Candidate to Leader 01:51:58,840 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderPayloadVersion=13] 01:51:58,841 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:58,841 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:58,841 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from Candidate to Leader 01:51:58,841 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: All Shards are ready - data store testTransactionChainWithSingleShard is ready 01:51:58,842 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithSingleShard: Closing 01:51:58,842 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testTransactionChainWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 01:51:58,842 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderPayloadVersion=13] 01:51:58,842 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:58,842 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:58,843 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: All Shards are ready - data store testTransactionChainWithSingleShard is ready 01:51:58,843 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard Received follower initial sync status for member-2-shard-cars-testTransactionChainWithSingleShard status sync done false 01:51:58,847 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: -1, nextIndex: 0 01:51:58,847 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:51:58,864 PM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1cb27634 01:51:58,864 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1cb27634 changed state from IDLE to TX_OPEN 01:51:58,864 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} 01:51:58,864 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:58,864 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:58,864 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#2060711106] 01:51:58,864 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} shard 1 01:51:58,864 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:51:58,864 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:51:58,864 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 01:51:58,864 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@4cd8cb91 01:51:58,864 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1cb27634 changed state from TX_OPEN to IDLE 01:51:58,865 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1cb27634 readied transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:51:58,865 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 01:51:58,865 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 0 01:51:58,865 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@4ab69830 01:51:58,865 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:58,865 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard, primaryVersion=13] 01:51:58,865 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard, primaryVersion=13] 01:51:58,866 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@74908866, attempting to connect 01:51:58,868 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} is not yet known 01:51:58,868 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, nanosAgo=10111, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:51:58,869 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:51:58,869 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:58,869 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:58,870 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@59df716c instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@3ee5ffeb 01:51:58,870 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state SEALED 01:51:58,870 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} previous state OPEN 01:51:58,870 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:58,870 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@27be494e transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111]}, enqueuedTicks=873195256056} 01:51:58,870 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@3ee5ffeb 01:51:58,870 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:51:58,870 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} 01:51:58,870 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=1, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} 01:51:58,870 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 01:51:58,870 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2238/0x00000008016aba80@2ba00d8f 01:51:58,870 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@27be494e transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=1, protocol=READY}, enqueuedTicks=873195486971} 01:51:58,870 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-689: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=1, protocol=READY}} 01:51:58,870 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-689: Message size: 390 bytes, total slices to send: 1 01:51:58,870 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-689: Message does not need to be sliced - sending original message 01:51:58,870 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} 01:51:58,870 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@3ee5ffeb 01:51:58,870 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 01:51:58,870 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} 01:51:58,870 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} 01:51:58,871 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 0 01:51:58,871 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1247/0x0000000801301400@82ae7e8 01:51:58,871 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@27be494e transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=873195740178} 01:51:58,871 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1cb27634 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:58,871 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 01:51:58,871 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} 01:51:58,871 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@2867e83b 01:51:58,871 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@65259460 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@e61b414, splicing queue 01:51:58,871 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@65259460 queue spliced 0 messages 01:51:58,871 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.309 ms 01:51:58,873 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 01:51:58,873 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 01:51:58,873 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:51:58,873 PM [cluster-test-shard-dispatcher-23] [DEBUG] LeaderFrontendState - member-1-shard-cars-testTransactionChainWithSingleShard: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:51:58,873 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 01:51:58,874 PM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: allocating new open transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:51:58,874 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:51:58,874 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@4ef02a51 01:51:58,875 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:51:58,875 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 ready 01:51:58,875 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE}}}]} 01:51:58,875 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: transitioned member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 to ready 01:51:58,875 PM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: allocating new open snapshot member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 01:51:58,875 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Reusing a chained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:51:58,875 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 snapshot MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE}}}] 01:51:58,875 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-688: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0}} 01:51:58,875 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-688: Message size: 331 bytes, total slices to send: 1 01:51:58,875 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-688: Message does not need to be sliced - sending original message 01:51:58,876 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0}} 01:51:58,876 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=1, protocol=READY}, enqueuedTicks=873195486971, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0}} 01:51:58,876 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0} 01:51:58,876 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0}} 01:51:58,876 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=873195740178, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0}} 01:51:58,876 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0} 01:51:58,876 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1cb27634 changed state from IDLE to TX_OPEN 01:51:58,876 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:51:58,876 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 0 01:51:58,876 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@2ee881cc 01:51:58,876 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@27be494e transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=873207207622} 01:51:58,877 PM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: allocating new open transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:51:58,877 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Reusing a chained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:51:58,877 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 snapshot MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}] 01:51:58,877 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-688: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0}} 01:51:58,877 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-688: Message size: 331 bytes, total slices to send: 1 01:51:58,877 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-688: Message does not need to be sliced - sending original message 01:51:58,878 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0}} 01:51:58,878 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=873207207622, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0}} 01:51:58,878 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0} 01:51:58,878 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 1 01:51:58,878 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 2 01:51:58,878 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=2} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@268a1200 01:51:58,878 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@27be494e transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=2}, enqueuedTicks=873209031047} 01:51:58,878 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-689: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=2}} 01:51:58,878 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-689: Message size: 460 bytes, total slices to send: 1 01:51:58,878 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-689: Message does not need to be sliced - sending original message 01:51:58,878 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ExistsTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1239/0x0000000801302f10@6e01996b 01:51:58,878 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@27be494e transmitting entry ConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]}, enqueuedTicks=873209286133} 01:51:58,882 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=2} with Envelope{sessionId=0, txSequence=4, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1}} 01:51:58,882 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=2}, enqueuedTicks=873209031047, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1}} 01:51:58,882 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=2} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1} 01:51:58,882 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ExistsTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]} with Envelope{sessionId=0, txSequence=5, message=ExistsTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, exists=true}} 01:51:58,882 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]}, enqueuedTicks=873209286133, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=ExistsTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, exists=true}} 01:51:58,882 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Exists request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] completed with ExistsTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, exists=true} 01:51:58,882 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 3 01:51:58,882 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@1ec9e26f 01:51:58,882 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@27be494e transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=873213149929} 01:51:58,883 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-688: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3}} 01:51:58,883 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-688: Message size: 370 bytes, total slices to send: 1 01:51:58,883 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-688: Message does not need to be sliced - sending original message 01:51:58,884 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3}} 01:51:58,884 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=873213149929, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3}} 01:51:58,884 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3} 01:51:58,884 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 4 01:51:58,884 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 01:51:58,884 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@2dd45f25 01:51:58,884 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@27be494e transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=0, protocol=SIMPLE}, enqueuedTicks=873215126728} 01:51:58,884 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-689: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=0, protocol=SIMPLE}} 01:51:58,884 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-689: Message size: 286 bytes, total slices to send: 1 01:51:58,884 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-689: Message does not need to be sliced - sending original message 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 is already in state READY (READY) 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 initiating direct canCommit 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Validating transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 validated 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 initiating direct preCommit 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Preparing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 candidate ready 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1: canCommit - candidate rootPath: / 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1: canCommit - messages: [] 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1: preCommit - successfulFromPrevious: [] 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 prepared 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 initiating direct commit 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Starting commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 174 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}] 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}] 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:51:58,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 01:51:58,886 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}]]] 01:51:58,886 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 submitted to persistence 01:51:58,888 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}] 01:51:58,888 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}]]] 01:51:58,888 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 2 01:51:58,888 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:51:58,888 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 01:51:58,888 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 01:51:58,889 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log Simple{size=174} 01:51:58,889 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}] 01:51:58,889 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 2 01:51:58,889 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:58,889 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 01:51:58,889 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}] 01:51:58,889 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 1, nextIndex: 2 01:51:58,889 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 01:51:58,889 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 0 01:51:58,889 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from -1 to 0 01:51:58,889 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard Received follower initial sync status for member-2-shard-cars-testTransactionChainWithSingleShard status sync done true 01:51:58,889 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 01:51:58,889 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2241/0x00000008016af5d0@37375567 01:51:58,889 PM [cluster-test-shard-dispatcher-23] [DEBUG] LeaderFrontendState - member-1-shard-cars-testTransactionChainWithSingleShard: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:51:58,889 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionChainWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1853a7c7}, purged=MutableUnsignedLongSet{size=0}} 01:51:58,889 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:51:58,889 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 1 01:51:58,889 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 0 to 1 01:51:58,889 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 1 data Simple{size=174} 01:51:58,889 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Resuming commit of transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:51:58,889 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Committed transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:51:58,889 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1: commit - successfulFromPrevious: [] 01:51:58,889 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:51:58,889 PM [cluster-test-shard-dispatcher-23] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 01:51:58,889 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionChainWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-67065980] with name member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher 01:51:58,889 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-67065980] 01:51:58,890 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 0 for term 4 01:51:58,890 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 01:51:58,890 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:51:58,890 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=1 01:51:58,890 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 54.71 μs 01:51:58,890 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-67065980] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:58,890 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} 01:51:58,890 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 1 01:51:58,891 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111]}, enqueuedTicks=873195256056, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} 01:51:58,891 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 01:51:58,891 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0} 01:51:58,891 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 0 01:51:58,891 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from -1 to 0 01:51:58,891 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=7, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1}} 01:51:58,891 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 01:51:58,891 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=0, protocol=SIMPLE}, enqueuedTicks=873215126728, sessionId=0, txSequence=7} with Envelope{sessionId=0, txSequence=7, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1}} 01:51:58,891 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} directCommit completed 01:51:58,891 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED}: initiating purge 01:51:58,891 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 2 01:51:58,891 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-290863111]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@51ab1797 01:51:58,891 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionChainWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@2d692787}, purged=MutableUnsignedLongSet{size=0}} 01:51:58,891 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:51:58,891 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:58,891 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 1 01:51:58,891 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 0 to 1 01:51:58,891 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 1 data Simple{size=174} 01:51:58,891 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@27be494e transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-290863111]}, enqueuedTicks=873221652579} 01:51:58,891 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithSingleShard: Applying foreign transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:51:58,891 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@53d48d84 01:51:58,891 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@27be494e transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=2, protocol=READY}, enqueuedTicks=873221846884} 01:51:58,891 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-689: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=9, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=2, protocol=READY}} 01:51:58,891 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-689: Message size: 474 bytes, total slices to send: 1 01:51:58,891 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-689: Message does not need to be sliced - sending original message 01:51:58,891 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1cb27634 changed state from TX_OPEN to IDLE 01:51:58,891 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1cb27634 readied transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:51:58,891 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=FLUSHED} allocated sequence 5 01:51:58,891 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Committed transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:51:58,891 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@b8de0e4 01:51:58,891 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@27be494e transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=0, protocol=SIMPLE}, enqueuedTicks=873222165402} 01:51:58,891 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-689: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=a, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=0, protocol=SIMPLE}} 01:51:58,891 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-689: Message size: 287 bytes, total slices to send: 1 01:51:58,891 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-689: Message does not need to be sliced - sending original message 01:51:58,891 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionChainWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-803491137] with name member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher 01:51:58,891 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-803491137] 01:51:58,891 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 0 for term 4 01:51:58,891 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 01:51:58,891 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=1 01:51:58,891 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 39.20 μs 01:51:58,891 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-803491137] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:58,892 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: purging transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:51:58,892 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 01:51:58,892 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 01:51:58,892 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 ready 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=MERGE, modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}], operation=MERGE}}}}}}}]} 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: transitioned member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 to ready 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 is already in state READY (READY) 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 initiating direct canCommit 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Validating transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 validated 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 initiating direct preCommit 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Preparing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 candidate ready 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1: canCommit - candidate rootPath: / 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1: canCommit - messages: [] 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1: preCommit - successfulFromPrevious: [] 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 prepared 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 initiating direct commit 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Starting commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 230 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}] 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}] 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:51:58,893 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 01:51:58,894 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 01:51:58,894 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}]]] 01:51:58,894 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 submitted to persistence 01:51:58,894 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 01:51:58,894 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}] 01:51:58,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}]]] 01:51:58,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 2 01:51:58,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:51:58,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66} 01:51:58,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 01:51:58,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log Simple{size=230} 01:51:58,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}] 01:51:58,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 3 01:51:58,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:58,894 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=2, protocol=READY} with Envelope{sessionId=0, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4}} 01:51:58,894 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=2, protocol=READY}, enqueuedTicks=873221846884, sessionId=0, txSequence=9} with Envelope{sessionId=0, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4}} 01:51:58,894 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=2, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4} 01:51:58,895 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 01:51:58,895 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}] 01:51:58,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 3, nextIndex: 4 01:51:58,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 01:51:58,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 2 01:51:58,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 1 to 2 01:51:58,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66} 01:51:58,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@12666803 01:51:58,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: finished purging transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:51:58,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Purged transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:51:58,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 3 01:51:58,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 2 to 3 01:51:58,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 3 data Simple{size=230} 01:51:58,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Resuming commit of transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:51:58,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Committed transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:51:58,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1: commit - successfulFromPrevious: [] 01:51:58,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 changing state from COMMIT_PENDING to COMMITTED 01:51:58,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=MERGE, modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}], operation=MERGE, modificationType=WRITE}}}}}}}]} 01:51:58,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 2 for term 4 01:51:58,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:51:58,896 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=3 01:51:58,897 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 01:51:58,897 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 41.55 μs 01:51:58,897 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 3 01:51:58,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 01:51:58,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 2 01:51:58,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 1 to 2 01:51:58,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66} 01:51:58,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Purged transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 01:51:58,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 3 01:51:58,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 2 to 3 01:51:58,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 3 data Simple{size=230} 01:51:58,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithSingleShard: Applying foreign transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:51:58,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Committed transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:51:58,898 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 01:51:58,898 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 40.43 μs 01:51:58,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 2 for term 4 01:51:58,899 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-290863111]} with Envelope{sessionId=0, txSequence=8, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2}} 01:51:58,899 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=3 01:51:58,899 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-290863111]}, enqueuedTicks=873221652579, sessionId=0, txSequence=8} with Envelope{sessionId=0, txSequence=8, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2}} 01:51:58,899 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 01:51:58,899 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@3ee5ffeb purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} 01:51:58,899 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5}} 01:51:58,899 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=0, protocol=SIMPLE}, enqueuedTicks=873222165402, sessionId=0, txSequence=10} with Envelope{sessionId=0, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5}} 01:51:58,899 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=FLUSHED} directCommit completed 01:51:58,899 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=FLUSHED}: initiating purge 01:51:58,899 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=DONE} allocated sequence 6 01:51:58,899 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-290863111]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@2f1203ad 01:51:58,899 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:58,899 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@27be494e transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-290863111]}, enqueuedTicks=873229710119} 01:51:58,899 PM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 01:51:58,899 PM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 01:51:58,899 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@59df716c aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} 01:51:58,899 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} allocated sequence 1 01:51:58,899 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@fda6e65 01:51:58,899 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} 01:51:58,899 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SEALED} allocated sequence 1 01:51:58,899 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SEALED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1240/0x0000000801303148@41294ae7 01:51:58,899 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@27be494e transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=0, protocol=ABORT}, enqueuedTicks=873230088308} 01:51:58,899 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-689: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=c, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=0, protocol=ABORT}} 01:51:58,899 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-689: Message size: 287 bytes, total slices to send: 1 01:51:58,899 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-689: Message does not need to be sliced - sending original message 01:51:58,899 PM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 01:51:58,899 PM [main] [WARN] ClientBackedTransactionChain - Aborted unclosed transactions [member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0] java.lang.Throwable: at at org.opendaylight.controller.cluster.databroker.ClientBackedTransactionChain.close(ClientBackedTransactionChain.java:76) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.testTransactionChainWithSingleShard(DistributedDataStoreRemotingIntegrationTest.java:581) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) 01:51:58,899 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@3ee5ffeb invoking destroy 01:51:58,900 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@27be494e transmitting entry ConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111]}, enqueuedTicks=873230548380} 01:51:58,900 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1cb27634 changed state from IDLE to CLOSED 01:51:58,900 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@30a4067b: creating a new snapshot member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 01:51:58,900 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=0, cookie=0} shard 1 01:51:58,900 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 01:51:58,900 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:51:58,900 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@632cddc1 01:51:58,900 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@27be494e transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=873230816577} 01:51:58,900 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: purging transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:51:58,900 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 01:51:58,900 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 01:51:58,900 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:51:58,901 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: aborting transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 01:51:58,901 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 01:51:58,901 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 01:51:58,901 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 01:51:58,901 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: closing history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:51:58,901 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:51:58,901 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 01:51:58,901 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 01:51:58,901 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CloseLocalHistoryPayload, isSendImmediate: false 01:51:58,901 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: allocating new open snapshot member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1 01:51:58,901 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-688: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=e, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0}} 01:51:58,901 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-688: Message size: 369 bytes, total slices to send: 1 01:51:58,901 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-688: Message does not need to be sliced - sending original message 01:51:58,901 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 01:51:58,902 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 01:51:58,902 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 01:51:58,902 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=e, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0}} 01:51:58,902 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-290863111], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=873230816577, sessionId=0, txSequence=14} with Envelope{sessionId=0, txSequence=e, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0}} 01:51:58,902 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0} 01:51:58,902 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:51:58,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testTransactionChainWithSingleShard 01:51:58,903 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithSingleShard - waiting on 1 shards 01:51:58,903 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onShutDown 01:51:58,903 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Initiating leader transfer 01:51:58,903 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@65d09113 01:51:58,903 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:51:58,903 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@65d09113 01:51:58,903 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Attempting to transfer leadership 01:51:58,903 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:51:58,903 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 6, leaderSnapShotIndex: 2 01:51:58,903 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 01:51:58,903 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}], SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}], SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}]]] 01:51:58,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:58,903 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:51:58,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}], SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}], SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}]]] 01:51:58,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 3 01:51:58,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 01:51:58,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67} 01:51:58,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 01:51:58,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67} 01:51:58,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 01:51:58,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 01:51:58,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 01:51:58,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 4 01:51:58,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:51:58,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 01:51:58,904 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 6, nextIndex: 7 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 3 01:51:58,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 4 01:51:58,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 3 to 4 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67} 01:51:58,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard] 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@20f61261 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: finished purging transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:51:58,905 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Purged transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:51:58,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 5 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 4 to 5 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 5 data AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67} 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2284/0x00000008016d86d0@109a0346 01:51:58,905 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Aborted transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 6 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 5 to 6 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 6 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 01:51:58,905 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@27be494e superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@1f839554, splicing queue 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2255/0x00000008016bd008@39d049fc 01:51:58,905 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:51:58,905 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@27be494e queue spliced 3 messages 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 5 for term 4 01:51:58,905 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:58,905 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: refreshing backend for shard 1 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 01:51:58,905 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:51:58,905 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:51:58,905 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionChainWithSingleShard, matchIndex: 6, lastIndex: 6, isVoting: true 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 01:51:58,905 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#2060711106] 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Leader transfer complete 01:51:58,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=6 01:51:58,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:51:58,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:51:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 6 01:51:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 3, lastIndex(): 6 01:51:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 4 01:51:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 3 to 4 01:51:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67} 01:51:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Purged transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 01:51:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 5 01:51:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 4 to 5 01:51:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 5 data AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67} 01:51:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Aborted transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 01:51:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 6 01:51:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 5 to 6 01:51:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 6 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 01:51:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 01:51:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 5 for term 4 01:51:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=6 01:51:58,907 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6}} not found in inflight queue, checking pending queue 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-290863111]} with Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6}} 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-290863111]}, enqueuedTicks=873229710119, sessionId=0, txSequence=11} with Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6}} 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=DONE}: purge completed 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@3ee5ffeb purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=DONE} 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=c, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1}} not found in inflight queue, checking pending queue 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=c, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1}} 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=0, protocol=ABORT}, enqueuedTicks=873230088308, sessionId=0, txSequence=12} with Envelope{sessionId=0, txSequence=c, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1}} 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=0, protocol=ABORT} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1} 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1} 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR}: initiating purge 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} allocated sequence 2 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-290863111]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@7ad0181 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SEALED}: initiating purge 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=DONE} allocated sequence 2 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-290863111]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@4871d15 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=d, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1}} not found in inflight queue, checking pending queue 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Completing request DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111]} with Envelope{sessionId=0, txSequence=d, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1}} 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111]}, enqueuedTicks=873230548380, sessionId=0, txSequence=13} with Envelope{sessionId=0, txSequence=d, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1}} 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@3ee5ffeb destroy completed with LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1} 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@1cb27634: removed destroyed proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@3ee5ffeb 01:51:58,908 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:51:59,317 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: All shards shutdown - sending PoisonPill to self 01:51:59,317 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] ShardManager - shard-manager-testTransactionChainWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:51:59,317 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionChainWithSingleShard 01:51:59,317 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard (Leader)_slicer-id-691: Closing 01:51:59,317 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Aborting 0 pending queued transactions 01:51:59,317 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithSingleShard 01:51:59,317 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.SendHeartBeat] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:59,318 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 01:51:59,318 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 01:51:59,318 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1939924251] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1939924251] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:59,318 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 01:51:59,318 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: shutting down 01:51:59,318 PM [cluster-test-akka.actor.default-dispatcher-32] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 01:51:59,318 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1939924251] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1939924251] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:59,318 PM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1939924251] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1939924251] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:59,319 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1939924251] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1939924251] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:51:59,319 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:59,319 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/11/2024 13:51:59.324] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1037)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:51:59.324] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1037)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:51:59.324] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1037)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:51:59.324] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1037)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:51:59,339 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithSingleShard: Closing 01:51:59,339 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionChainWithSingleShard 01:51:59,339 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Closing 01:51:59,339 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Aborting 0 pending queued transactions 01:51:59,339 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithSingleShard 01:51:59,340 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:51:59,340 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:00,338 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-7818456595557589733] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:52:00,357 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:00,357 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:00,358 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:52:00.359] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:00,360 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [277931340935473324] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:52:00,376 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:00,377 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:00,378 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:52:00.380] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:00,396 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:52:00,400 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [7752462888536930962] 01:52:00,401 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:52:00,402 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:52:00,402 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:52:00,410 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:00,410 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:00,411 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:52:00,411 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:52:00,411 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:52:00,414 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:52:00,418 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-6988920721045703250] 01:52:00,418 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:52:00,426 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:00,426 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:52:00,426 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:00,426 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:00,438 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:52:00,445 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-6293795995095640786] 01:52:00,445 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:52:00,446 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:00,446 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:52:00,446 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:00,447 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:00,457 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:00,457 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:00,458 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:52:00,463 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:00,463 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:00,464 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection closed with error: Connection reset 01:52:00,464 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:52:00.465] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:00,472 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 01:52:00,472 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 01:52:00,490 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:00,490 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:00,491 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:52:00.492] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:00,508 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:52:00,514 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-7833830417817871474] 01:52:00,514 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:52:00,516 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:52:00,516 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:52:00,526 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:00,526 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:00,526 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:52:00,526 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:52:00,526 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:52:00,530 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:52:00,536 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-505062915899253411] 01:52:00,536 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:52:00,536 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:00,536 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:52:00,542 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:00,542 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:00,553 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:52:00,569 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-8086049717915887581] 01:52:00,569 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:52:00,571 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:00,571 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:52:00,576 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:00,576 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:00,583 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:00,583 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@329a5ca3 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} 01:52:00,583 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 1 01:52:00,583 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-290863111], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@3b123593 01:52:00,583 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-634698518], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:00,583 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0} 01:52:00,585 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:00,587 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testForwardedReadyTransactionForwardedToLeader 01:52:00,595 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-2146459137] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:00,596 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 01:52:00,601 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: no initial generation override, starting from 0 01:52:00,601 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 01:52:00,601 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4be7b2dc to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6b5b6cc2 01:52:00,601 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2e59efcd 01:52:00,601 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, 0, 1710165120601, None)) 01:52:00,601 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testForwardedReadyTransactionForwardedToLeader 01:52:00,601 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165120600,0,0)) 01:52:00,601 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Creating local shard: member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:00,602 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6b5b6cc2 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5a6d2b52 01:52:00,602 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:52:00,602 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} started 01:52:00,602 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:00,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3bb89ba3 01:52:00,602 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:00,603 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:52:00,603 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:00,603 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:00,604 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:00,604 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: updated peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903] 01:52:00,604 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:00,604 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, persistent : true 01:52:00,604 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:00,605 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-1579247071 created and ready for shard:member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:00,605 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-1579247071] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:00,605 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-1:snapshot-read#-1761344454] 01:52:00,605 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-1:snapshot-read#-1761344454] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:00,605 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with journal batch size 1 01:52:00,605 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testForwardedReadyTransactionForwardedToLeader 01:52:00,607 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:52:00,607 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:52:00,607 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:52:00,607 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 01:52:00,607 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Initializing shard [cars] 01:52:00,607 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is initialized 01:52:00,607 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from null to Follower 01:52:00,611 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader 01:52:00,620 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:52:00,622 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:52:00,628 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from null to Follower 01:52:00,629 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 01:52:00,629 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#1363277257] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:00,651 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testForwardedReadyTransactionForwardedToLeader 01:52:00,651 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Creating local shard: member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:00,651 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:52:00,651 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:00,652 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: no initial generation override, starting from 0 01:52:00,652 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 01:52:00,652 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@63d57374 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2c88b335 01:52:00,652 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5d53c76f 01:52:00,652 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, 0, 1710165120652, None)) 01:52:00,652 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165120651,0,0)) 01:52:00,652 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6427730b 01:52:00,652 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2c88b335 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4ca74c7a 01:52:00,653 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} started 01:52:00,653 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:52:00,653 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:00,655 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:00,655 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:00,655 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: updated peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1760513130] 01:52:00,655 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, persistent : true 01:52:00,655 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1760513130] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:00,656 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#1158926774 created and ready for shard:member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:00,656 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with journal batch size 1 01:52:00,656 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#1158926774] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:00,657 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:52:00,657 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:52:00,657 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Initializing shard [cars] 01:52:00,657 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader is initialized 01:52:00,659 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-2:snapshot-read#1226322211] 01:52:00,659 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-2:snapshot-read#1226322211] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:00,662 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:52:00,662 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 01:52:00,662 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:00,662 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from null to Follower 01:52:00,662 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader 01:52:00,662 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader from null to Follower 01:52:00,669 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:52:00,669 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:52:00,703 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-2146459137] 01:52:00,704 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 01:52:00,704 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903] 01:52:00,916 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:52:00,916 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Received ElectionTimeout - switching to Candidate 01:52:00,916 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] 01:52:00,916 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:00,916 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 1 01:52:00,916 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:52:00,916 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Closing 01:52:00,916 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:52:00,916 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 01:52:00,916 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from Follower to Candidate 01:52:00,916 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from Follower to Candidate 01:52:01,216 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:52:01,216 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,216 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 2 01:52:01,267 PM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:52:01,446 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:52:01,446 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,446 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 3 01:52:01,537 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:52:01,537 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:52:01,538 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:52:01,538 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,538 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,538 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:52:01,538 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,538 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,538 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,545 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:52:01,545 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,545 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,545 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:52:01,545 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,545 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,587 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:52:01,736 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 01:52:01,736 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,736 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 4 01:52:01,736 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,737 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:01,737 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Cluster state: [] 01:52:01,737 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:52:01,737 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:52:01,737 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=null 01:52:01,737 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:52:01,737 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,737 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:52:01,738 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:52:01,738 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:52:01,739 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] 01:52:01,739 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:52:01,739 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:52:01,739 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:52:01,739 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: became leader with frontend state for [] 01:52:01,739 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from Candidate to Leader 01:52:01,739 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderPayloadVersion=13] 01:52:01,739 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:01,739 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:52:01,739 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from Candidate to Leader 01:52:01,739 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All Shards are ready - data store testForwardedReadyTransactionForwardedToLeader is ready 01:52:01,745 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Closing 01:52:01,745 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,745 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader status sync done false 01:52:01,745 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderPayloadVersion=13] 01:52:01,745 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:01,745 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:52:01,745 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All Shards are ready - data store testForwardedReadyTransactionForwardedToLeader is ready 01:52:01,745 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: -1, nextIndex: 0 01:52:01,745 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:52:01,819 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#1363277257] 01:52:01,819 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 01:52:01,819 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1760513130] 01:52:01,820 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,820 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#1363277257] 01:52:01,820 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 01:52:01,820 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1760513130] 01:52:01,831 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: handleForwardedReadyTransaction for member-1-ShardTransactionTest-fe-0-txn-275-0 01:52:01,831 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,831 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader)] 01:52:01,833 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, disabling transaction tracking 01:52:01,833 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:52:01,833 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:52:01,833 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 01:52:01,833 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:52:01,833 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,833 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 01:52:01,833 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying 3 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-275-0 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-275-0 of 3 operations, client version 13 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-275-0 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-275-0 ready 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-275-0 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 validated 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-275-0: success 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-275-0 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-275-0 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 candidate ready 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-275-0: canCommit - candidate rootPath: / 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-275-0: canCommit - messages: [] 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-275-0: preCommit - successfulFromPrevious: [] 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 prepared 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-275-0 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-275-0 01:52:01,834 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 01:52:01,834 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:52:01,834 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 198 01:52:01,834 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}] 01:52:01,834 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}] 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-275-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:52:01,834 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 1 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:52:01,834 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,834 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 submitted to persistence 01:52:01,835 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:52:01,835 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader status sync done true 01:52:01,835 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 01:52:01,835 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}] 01:52:01,835 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 0, nextIndex: 1 01:52:01,835 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:52:01,835 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 0 01:52:01,835 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from -1 to 0 01:52:01,835 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:52:01,835 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 01:52:01,835 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 01:52:01,835 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, nanosAgo=2438881} is already disabled 01:52:01,835 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:52:01,835 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,836 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}]]] 01:52:01,836 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 01:52:01,836 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}]]] 01:52:01,836 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:52:01,836 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:52:01,836 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log Simple{size=198} 01:52:01,836 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}] 01:52:01,836 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 2 01:52:01,836 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 0 01:52:01,836 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:52:01,836 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:52:01,836 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 0 01:52:01,836 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from -1 to 0 01:52:01,837 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 01:52:01,837 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 01:52:01,837 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 01:52:01,837 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=198}] 01:52:01,837 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 01:52:01,837 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 1, nextIndex: 2 01:52:01,837 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:52:01,837 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 1 01:52:01,837 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 0 to 1 01:52:01,837 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=198} 01:52:01,837 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-275-0 01:52:01,837 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-275-0: commit - successfulFromPrevious: [] 01:52:01,837 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 changing state from COMMIT_PENDING to COMMITTED 01:52:01,837 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-275-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/system/testProbe-765#-1230985140] 01:52:01,838 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#991437053] with name member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher 01:52:01,838 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 01:52:01,838 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:52:01,838 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 01:52:01,838 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#991437053] 01:52:01,838 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 01:52:01,838 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#991437053] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:01,838 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 55.55 μs 01:52:01,838 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4ff4b602: creating a new snapshot member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 01:52:01,839 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} 01:52:01,839 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:01,839 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:01,839 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-2146459137] 01:52:01,839 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:01,839 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, history=0, cookie=0} shard 1 01:52:01,839 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6befd480] 01:52:01,839 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1 for transaction member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 01:52:01,839 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:52:01,839 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6befd480] 01:52:01,839 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-264245909], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@5be290cc 01:52:01,839 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:52:01,839 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@7f7662d2, attempting to connect 01:52:01,839 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: client ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} is not yet known 01:52:01,839 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, nanosAgo=5970, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 01:52:01,839 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:52:01,839 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:52:01,839 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 1 01:52:01,839 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:52:01,839 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:01,839 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 1 01:52:01,839 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 0 to 1 01:52:01,839 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=198} 01:52:01,839 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5f9cafb6 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3cba15e0 01:52:01,840 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:52:01,840 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3cba15e0 01:52:01,840 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1 for transaction member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1 01:52:01,840 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-275-0 01:52:01,840 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 01:52:01,840 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-264245909], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 01:52:01,840 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4ff4b602 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:01,840 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} 01:52:01,840 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@d4c6c41 01:52:01,840 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@30ecec47 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@5c95eaaf, splicing queue 01:52:01,840 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@30ecec47 queue spliced 0 messages 01:52:01,840 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 344.0 μs 01:52:01,840 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#368979105] with name member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher 01:52:01,840 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0} 01:52:01,840 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#368979105] 01:52:01,840 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 01:52:01,840 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 01:52:01,840 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 01:52:01,840 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 87.66 μs 01:52:01,840 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#368979105] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:01,841 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: handleForwardedReadyTransaction for member-1-ShardTransactionTest-fe-0-txn-276-0 01:52:01,841 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,841 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader)] 01:52:01,841 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-276-0 01:52:01,841 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-276-0 of 1 operations, client version 13 01:52:01,841 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-276-0 01:52:01,841 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-276-0 ready 01:52:01,842 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0 canCommit 01:52:01,842 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903)] resolved 01:52:01,842 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: successfully resolved all cohort actors 01:52:01,842 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0 finishCanCommit 01:52:01,842 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: sending CanCommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-276-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903)] 01:52:01,842 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903)] 01:52:01,842 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Can committing transaction member-1-ShardTransactionTest-fe-0-txn-276-0 01:52:01,842 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-276-0 01:52:01,842 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 validated 01:52:01,842 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:52:01,842 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-276-0: success 01:52:01,842 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: received CanCommitTransactionReply [canCommit=true, version=13] 01:52:01,842 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: canCommit returning result: true 01:52:01,842 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0 commit 01:52:01,842 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0 finish commit 01:52:01,842 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: Sending CommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-276-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903)] 01:52:01,842 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903)] 01:52:01,843 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-276-0 01:52:01,843 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-276-0 01:52:01,843 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 candidate ready 01:52:01,843 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-276-0: canCommit - candidate rootPath: / 01:52:01,843 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-276-0: canCommit - messages: [] 01:52:01,843 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-276-0: preCommit - successfulFromPrevious: [] 01:52:01,843 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 prepared 01:52:01,843 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:52:01,843 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-276-0 01:52:01,843 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-276-0 01:52:01,843 PM [cluster-test-shard-dispatcher-25] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 209 01:52:01,843 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}] 01:52:01,843 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}] 01:52:01,843 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-276-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:52:01,843 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 01:52:01,843 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,843 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}]]] 01:52:01,843 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 submitted to persistence 01:52:01,843 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}] 01:52:01,844 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}]]] 01:52:01,844 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 01:52:01,844 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:52:01,844 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log Simple{size=209} 01:52:01,844 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}] 01:52:01,844 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 2 01:52:01,844 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:52:01,844 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=209}] 01:52:01,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 2, nextIndex: 3 01:52:01,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 01:52:01,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 2 01:52:01,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 1 to 2 01:52:01,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=209} 01:52:01,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-276-0 01:52:01,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-276-0: commit - successfulFromPrevious: [] 01:52:01,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 changing state from COMMIT_PENDING to COMMITTED 01:52:01,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-276-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testForwardedReadyTransactionForwardedToLeader_member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader%2395236903$a] 01:52:01,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 01:52:01,845 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:52:01,845 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 01:52:01,845 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-276-0: commit succeeded 01:52:01,845 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 01:52:01,845 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 41.22 μs 01:52:01,845 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4ff4b602: creating a new snapshot member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0 01:52:01,845 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-1 for transaction member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0 01:52:01,845 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 01:52:01,845 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Issuing gracefulStop to shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:01,845 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader - waiting on 1 shards 01:52:01,845 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onShutDown 01:52:01,845 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Initiating leader transfer 01:52:01,845 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 01:52:01,845 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1391418f 01:52:01,845 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:01,846 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:52:01,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1391418f 01:52:01,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Attempting to transfer leadership 01:52:01,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:52:01,846 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@525f7a46 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@175dfe69, splicing queue 01:52:01,846 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@525f7a46 queue spliced 0 messages 01:52:01,846 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:01,846 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: refreshing backend for shard 1 01:52:01,846 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:52:01,846 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:01,846 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:01,846 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-2146459137] 01:52:01,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:01,846 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 2 01:52:01,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 01:52:01,846 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 01:52:01,846 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 2 01:52:01,846 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 1 to 2 01:52:01,846 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=209} 01:52:01,846 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-276-0 01:52:01,846 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 01:52:01,846 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 01:52:01,847 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 01:52:01,847 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 78.63 μs 01:52:01,847 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, matchIndex: 2, lastIndex: 2, isVoting: true 01:52:01,847 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Follower's log matches - sending ElectionTimeout 01:52:01,847 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:52:01,847 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Leader transfer complete 01:52:01,848 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader] 01:52:01,848 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 01:52:01,848 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:01,848 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:52:02,266 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 01:52:02,266 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:52:02,266 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received ShardNotInitializedTimeout message for shard cars 01:52:02,266 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Returning NoShardLeaderException for shard cars 01:52:02,266 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:52:02,266 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Stopping Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:02,267 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader)_slicer-id-696: Closing 01:52:02,267 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:02,267 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Aborting 0 pending queued transactions 01:52:02,267 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:02,267 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 01:52:02,267 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:02,267 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 01:52:02,267 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 01:52:02,268 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 01:52:02,268 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: shutting down 01:52:02,268 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 01:52:02,269 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:02,269 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:02,274 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:02,274 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:02,274 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:02,274 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:02,286 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 01:52:02,287 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 01:52:02,287 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Closing 01:52:02,287 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Aborting 0 pending queued transactions 01:52:02,287 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 01:52:02,287 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:02,287 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:03,287 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-7833830417817871474] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:52:03,301 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:03,301 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:03,305 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:52:03.307] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:03,307 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-505062915899253411] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [03/11/2024 13:52:03.322] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:03,324 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:03,346 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:52:03,351 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-819116265572607807] 01:52:03,351 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:52:03,358 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:52:03,358 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:52:03,359 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:03,359 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:03,362 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:52:03,362 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:52:03,362 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:52:03,370 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:52:03,374 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [8278792735637792132] 01:52:03,374 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:52:03,378 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:03,378 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:52:03,378 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:03,378 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:03,389 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:52:03,394 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [340515682280265863] 01:52:03,394 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:52:03,404 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:52:03,405 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:03,405 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:52:03,406 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:52:03,406 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:03,406 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:03,416 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:03,427 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:52:03,428 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:03,429 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:52:03,429 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithSingleShard 01:52:03,432 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 01:52:03,434 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1544042174] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:03,435 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: no initial generation override, starting from 0 01:52:03,435 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:52:03,435 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2e94bbd1 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3d25dfd5 01:52:03,435 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@391a5ca9 01:52:03,436 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, 0, 1710165123436, None)) 01:52:03,436 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithSingleShard 01:52:03,436 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Creating local shard: member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:03,436 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165123435,0,0)) 01:52:03,436 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:52:03,436 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:03,436 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@30e04606 01:52:03,436 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3d25dfd5 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3db5bd36 01:52:03,440 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} started 01:52:03,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:52:03,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:03,440 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:03,440 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:03,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:03,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: updated peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard on shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1622918803] 01:52:03,441 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testChainedTransactionFailureWithSingleShard, persistent : true 01:52:03,441 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1622918803] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:03,441 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#1570458904 created and ready for shard:member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:03,441 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#1570458904] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:03,441 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-1:snapshot-read#-1081747405] 01:52:03,441 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testChainedTransactionFailureWithSingleShard with journal batch size 1 01:52:03,441 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-1:snapshot-read#-1081747405] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:03,441 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:03,442 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:52:03,442 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:52:03,442 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Initializing shard [cars] 01:52:03,442 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is initialized 01:52:03,442 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:52:03,442 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 01:52:03,442 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from null to Follower 01:52:03,442 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard 01:52:03,442 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from null to Follower 01:52:03,443 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithSingleShard 01:52:03,447 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#647629871] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:03,448 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 01:52:03,452 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: no initial generation override, starting from 0 01:52:03,452 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:52:03,452 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@78e5fb3 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1524bea2 01:52:03,453 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithSingleShard 01:52:03,453 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Creating local shard: member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:03,453 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:52:03,453 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:03,459 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@35208d04 01:52:03,459 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, 0, 1710165123453, None)) 01:52:03,459 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165123452,0,0)) 01:52:03,460 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1524bea2 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@36572a68 01:52:03,460 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@8e3ff8 01:52:03,463 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} started 01:52:03,463 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:52:03,463 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:03,464 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:03,464 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:03,464 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: updated peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard on shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard#-1755420155] 01:52:03,464 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testChainedTransactionFailureWithSingleShard, persistent : true 01:52:03,464 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testChainedTransactionFailureWithSingleShard with journal batch size 1 01:52:03,465 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard#-1755420155] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:03,465 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-2:snapshot-read#615183738] 01:52:03,465 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#311409032 created and ready for shard:member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:03,465 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-2:snapshot-read#615183738] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:03,465 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#311409032] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:03,466 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:52:03,466 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:52:03,466 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:52:03,466 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 01:52:03,466 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:03,466 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Initializing shard [cars] 01:52:03,466 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard is initialized 01:52:03,467 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from null to Follower 01:52:03,467 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard 01:52:03,467 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-2-shard-cars-testChainedTransactionFailureWithSingleShard from null to Follower 01:52:03,514 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1544042174] 01:52:03,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: findLocalShard : cars 01:52:03,514 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1622918803] 01:52:03,733 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:52:03,733 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 01:52:03,733 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testChainedTransactionFailureWithSingleShard] 01:52:03,733 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:03,733 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 1 01:52:03,733 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:52:03,733 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Closing 01:52:03,733 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 01:52:03,733 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 01:52:03,733 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from Follower to Candidate 01:52:03,733 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from Follower to Candidate 01:52:04,2 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 01:52:04,2 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:04,3 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 2 01:52:04,14 PM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:52:04,242 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 01:52:04,242 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:04,242 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 3 01:52:04,374 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:52:04,374 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:52:04,375 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:52:04,375 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:04,375 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:04,375 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:52:04,375 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:04,375 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:04,375 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:04,381 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:52:04,381 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:04,381 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:04,381 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:52:04,381 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:04,381 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:04,418 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:52:04,562 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 01:52:04,562 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:04,562 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 4 01:52:04,562 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:04,564 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:04,564 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Cluster state: [] 01:52:04,564 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:52:04,564 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:52:04,564 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=null 01:52:04,564 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:52:04,564 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:04,564 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:52:04,565 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:52:04,565 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:52:04,565 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testChainedTransactionFailureWithSingleShard] 01:52:04,565 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:52:04,565 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:52:04,565 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 01:52:04,565 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: became leader with frontend state for [] 01:52:04,565 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from Candidate to Leader 01:52:04,565 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderPayloadVersion=13] 01:52:04,566 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:04,566 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:52:04,566 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from Candidate to Leader 01:52:04,566 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All Shards are ready - data store testChainedTransactionFailureWithSingleShard is ready 01:52:04,566 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Closing 01:52:04,566 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testChainedTransactionFailureWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:04,566 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithSingleShard status sync done false 01:52:04,566 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderPayloadVersion=13] 01:52:04,566 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:04,566 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:52:04,567 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All Shards are ready - data store testChainedTransactionFailureWithSingleShard is ready 01:52:04,567 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: -1, nextIndex: 0 01:52:04,567 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:52:04,571 PM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@5798e5f5 01:52:04,571 PM [main] [DEBUG] AbstractDOMBroker - Transaction chain 0 created with listener Mock for DOMTransactionChainListener, hashCode: 447208024, backing store chains {CONFIGURATION=org.opendaylight.controller.cluster.databroker.ClientBackedTransactionChain@7447bbfc} 01:52:04,572 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@5798e5f5 changed state from IDLE to TX_OPEN 01:52:04,572 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} 01:52:04,572 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:04,572 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:04,572 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#647629871] 01:52:04,572 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0} shard 1 01:52:04,572 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:52:04,572 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 01:52:04,572 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 01:52:04,572 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1305193874], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2236/0x00000008016aaa68@424020fb 01:52:04,572 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@5798e5f5 changed state from TX_OPEN to IDLE 01:52:04,572 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@5798e5f5 readied transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 01:52:04,572 PM [main] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 is submitted for execution. 01:52:04,572 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 01:52:04,572 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1305193874], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@3e03e5c9 01:52:04,573 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:04,573 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard, primaryVersion=13] 01:52:04,573 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard, primaryVersion=13] 01:52:04,573 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@7a5df72b, attempting to connect 01:52:04,574 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} is not yet known 01:52:04,574 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, nanosAgo=8960, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:52:04,574 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:52:04,574 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1622918803], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:52:04,574 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1622918803], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:04,574 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@298083df instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@694dd96b 01:52:04,574 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:52:04,574 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1622918803], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:52:04,574 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@61cec209 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1305193874]}, enqueuedTicks=878903220547} 01:52:04,574 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@694dd96b 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1305193874], modifications=1, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1305193874], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2238/0x00000008016aba80@75489ce7 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@61cec209 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1305193874], modifications=1, protocol=READY}, enqueuedTicks=878903432922} 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-699: slice: identifier: member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1305193874], modifications=1, protocol=READY}} 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-699: Message size: 487 bytes, total slices to send: 1 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-699: Message does not need to be sliced - sending original message 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1305193874], modifications=0, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 1 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1305193874], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2277/0x00000008016c5c10@117f2c31 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@61cec209 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1305193874], modifications=0, protocol=SIMPLE}, enqueuedTicks=878903561555} 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-699: slice: identifier: member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1305193874], modifications=0, protocol=SIMPLE}} 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-699: Message size: 295 bytes, total slices to send: 1 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-699: Message does not need to be sliced - sending original message 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@5798e5f5 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1622918803], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@39573eb 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5006af4d superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@5e2ba002, splicing queue 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5006af4d queue spliced 0 messages 01:52:04,575 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1622918803], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 857.2 μs 01:52:04,576 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 01:52:04,576 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 01:52:04,576 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 01:52:04,576 PM [cluster-test-shard-dispatcher-22] [DEBUG] LeaderFrontendState - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 01:52:04,576 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: allocating new open transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 01:52:04,576 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 01:52:04,576 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@23f115db 01:52:04,576 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendTransaction - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1305193874], modifications=1, protocol=READY} failed to process org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:52:04,576 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: request Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1305193874], modifications=1, protocol=READY}} failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1305193874], modifications=1, protocol=READY} failed to process at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:147) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ... 33 more 01:52:04,577 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: request Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1305193874], modifications=0, protocol=SIMPLE}} failed org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:52:04,577 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 01:52:04,579 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1305193874], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1305193874], modifications=1, protocol=READY} failed to process}} 01:52:04,579 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1305193874], modifications=1, protocol=READY}, enqueuedTicks=878903432922, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1305193874], modifications=1, protocol=READY} failed to process}} 01:52:04,579 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1305193874], modifications=1, protocol=READY} completed with TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1305193874], modifications=1, protocol=READY} failed to process} 01:52:04,579 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1305193874], modifications=1, protocol=READY} failed to process at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:147) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ... 33 more 01:52:04,579 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1305193874], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=2, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 01:52:04,579 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1305193874], modifications=0, protocol=SIMPLE}, enqueuedTicks=878903561555, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 01:52:04,579 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1305193874], modifications=0, protocol=SIMPLE} completed with TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0} 01:52:04,579 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 failed org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:52:04,579 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during phase CAN_COMMIT, starting Abort org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:52:04,579 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DOMBrokerTransactionChain - Transaction chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@5016da36 failed. TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0]]} at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1119) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction.lambda$directCommit$4(AbstractProxyTransaction.java:517) at org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction.lambda$handleReplayedModifyTransactionRequest$16(RemoteProxyTransaction.java:509) at org.opendaylight.controller.cluster.access.client.ConnectionEntry.complete(ConnectionEntry.java:47) at org.opendaylight.controller.cluster.access.client.AbstractClientConnection.receiveResponse(AbstractClientConnection.java:494) at org.opendaylight.controller.cluster.access.client.AbstractReceivingClientConnection.receiveResponse(AbstractReceivingClientConnection.java:68) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onResponse(ClientActorBehavior.java:184) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onRequestFailure(ClientActorBehavior.java:196) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.internalOnRequestFailure(ClientActorBehavior.java:248) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:155) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:48) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.onReceiveCommand(AbstractClientActor.java:74) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) ... 17 more 01:52:04,581 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 aborted successfully 01:52:04,581 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} directCommit completed 01:52:04,581 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: initiating purge 01:52:04,581 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:52:04,581 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1305193874]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@4e80c9fa 01:52:04,581 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} 01:52:04,581 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED}: initiating purge 01:52:04,581 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 2 01:52:04,581 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1305193874]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@175c0b9f 01:52:04,581 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@61cec209 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1305193874]}, enqueuedTicks=878912027365} 01:52:04,582 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: purging transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 01:52:04,582 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}] 01:52:04,582 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}] 01:52:04,583 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:52:04,583 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}] 01:52:04,590 PM [main] [DEBUG] DOMBrokerTransactionChain - Chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@5016da36 is no longer running 01:52:04,590 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:52:04,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:04,591 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithSingleShard - waiting on 1 shards 01:52:04,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onShutDown 01:52:04,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Initiating leader transfer 01:52:04,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6424201c 01:52:04,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 01:52:04,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6424201c 01:52:04,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Attempting to transfer leadership 01:52:04,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:52:04,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:04,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}]]] 01:52:04,591 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:52:04,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:04,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:52:04,592 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard] 01:52:04,592 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=null, leaderPayloadVersion=13] 01:52:04,592 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}]]] 01:52:04,592 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Number of entries to be appended = 2 01:52:04,592 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:52:04,592 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74} 01:52:04,592 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:04,592 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 01:52:04,592 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75} 01:52:04,592 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}] 01:52:04,592 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:52:04,592 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Log size is now 2 01:52:04,592 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:52:04,592 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@61cec209 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@62b6598d, splicing queue 01:52:04,592 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1622918803], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:52:04,592 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@61cec209 queue spliced 2 messages 01:52:04,592 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1622918803], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1622918803], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:04,592 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: refreshing backend for shard 1 01:52:04,593 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1622918803], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:52:04,593 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:04,593 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:04,593 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#647629871] 01:52:04,593 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:04,593 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 01:52:04,593 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 01:52:04,593 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}] 01:52:04,593 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithSingleShard status sync done true 01:52:04,599 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: 1, nextIndex: 2 01:52:04,599 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 01:52:04,599 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Setting last applied to 0 01:52:04,599 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from -1 to 0 01:52:04,599 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74} 01:52:04,599 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2241/0x00000008016af5d0@7cf52f6 01:52:04,599 PM [cluster-test-shard-dispatcher-21] [DEBUG] LeaderFrontendState - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 01:52:04,599 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@23b4acc1}, purged=MutableUnsignedLongSet{size=0}} 01:52:04,599 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 01:52:04,599 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Setting last applied to 1 01:52:04,599 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from 0 to 1 01:52:04,599 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75} 01:52:04,599 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: replication of member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@1595790a 01:52:04,599 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: finished purging transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 01:52:04,599 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Purged transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 01:52:04,599 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: fakeSnapshot purging log to 0 for term 4 01:52:04,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:52:04,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testChainedTransactionFailureWithSingleShard, matchIndex: 1, lastIndex: 1, isVoting: true 01:52:04,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 01:52:04,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:52:04,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Leader transfer complete 01:52:04,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=1 01:52:04,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Commit index set to 1 01:52:04,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 01:52:04,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Setting last applied to 0 01:52:04,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from -1 to 0 01:52:04,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74} 01:52:04,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@147fa03c}, purged=MutableUnsignedLongSet{size=0}} 01:52:04,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 01:52:04,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Setting last applied to 1 01:52:04,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from 0 to 1 01:52:04,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75} 01:52:04,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Purged transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 01:52:04,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: fakeSnapshot purging log to 0 for term 4 01:52:04,601 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} not found in inflight queue, checking pending queue 01:52:04,601 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1305193874]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} 01:52:04,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=1 01:52:04,601 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1305193874]}, enqueuedTicks=878903220547, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} 01:52:04,601 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0} 01:52:04,601 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2}} not found in inflight queue, checking pending queue 01:52:04,601 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1305193874]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2}} 01:52:04,601 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1305193874]}, enqueuedTicks=878912027365, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2}} 01:52:04,601 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 01:52:04,601 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@694dd96b purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} 01:52:04,601 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: purge completed 01:52:04,601 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@298083df purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 01:52:04,803 PM [cluster-test-shard-dispatcher-21] [WARN] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 201, lastApplied : 1, commitIndex : 1 01:52:05,2 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All shards shutdown - sending PoisonPill to self 01:52:05,2 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1622918803]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:52:05,3 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:05,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader)_slicer-id-701: Closing 01:52:05,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Aborting 0 pending queued transactions 01:52:05,3 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 01:52:05,3 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 01:52:05,3 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 01:52:05,3 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: shutting down 01:52:05,3 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 01:52:05,4 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1544042174] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1544042174] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:05,4 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 01:52:05,4 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1544042174] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1544042174] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:05,4 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#-1889196075] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1889196075] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:05,4 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1544042174] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1544042174] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:05,4 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#-1889196075] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1889196075] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:05,4 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1544042174] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1544042174] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:05,4 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:05,4 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:05,8 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:05,8 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:05,8 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:05,8 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:05,10 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:52:05,25 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 01:52:05,26 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 01:52:05,26 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Closing 01:52:05,26 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Aborting 0 pending queued transactions 01:52:05,26 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 01:52:05,27 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:05,27 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:06,18 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-819116265572607807] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:52:06,44 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:06,44 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:06,45 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:52:06.046] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:06,48 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [8278792735637792132] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:52:06,55 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:06,55 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:06,56 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:52:06.057] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:06,73 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:52:06,77 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [7557809652491913287] 01:52:06,77 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:52:06,78 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:52:06,78 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:52:06,78 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:06,78 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:06,87 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:52:06,87 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:52:06,87 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:52:06,89 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:52:06,93 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-815793374557718875] 01:52:06,93 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:52:06,94 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:06,94 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:52:06,100 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:06,100 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:06,105 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:52:06,113 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-6047591842038498345] 01:52:06,113 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:52:06,118 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:06,118 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:52:06,122 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:06,123 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:06,123 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:06,123 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:06,124 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithShardLeaderNotResponding 01:52:06,142 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#1831734487] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:06,143 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:52:06,146 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:52:06,146 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 01:52:06,149 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:52:06,150 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: no initial generation override, starting from 0 01:52:06,150 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:52:06,150 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6a371a34 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@73131229 01:52:06,150 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@86f3ad 01:52:06,150 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, 0, 1710165126150, None)) 01:52:06,150 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithShardLeaderNotResponding 01:52:06,150 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Creating local shard: member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:06,151 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:52:06,151 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:06,151 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165126149,0,0)) 01:52:06,151 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@73131229 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@133ced66 01:52:06,151 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} started 01:52:06,151 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:06,153 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:06,153 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4c692c3b 01:52:06,153 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:52:06,153 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:06,153 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:06,153 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:06,153 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: updated peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding on shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-119000179] 01:52:06,154 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithShardLeaderNotResponding, persistent : true 01:52:06,154 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithShardLeaderNotResponding 01:52:06,154 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithShardLeaderNotResponding with journal batch size 1 01:52:06,155 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-119000179] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:06,155 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-1886639292 created and ready for shard:member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:06,155 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-1886639292] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:06,156 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:52:06,156 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:52:06,156 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-1:snapshot-read#1544053965] 01:52:06,156 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 01:52:06,156 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 01:52:06,156 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from null to Follower 01:52:06,156 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Initializing shard [cars] 01:52:06,156 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is initialized 01:52:06,156 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , registered listener akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding 01:52:06,157 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from null to Follower 01:52:06,157 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-1:snapshot-read#1544053965] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:06,157 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:52:06,157 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#1260191167] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:06,158 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 01:52:06,159 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithShardLeaderNotResponding 01:52:06,159 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Creating local shard: member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:06,159 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: no initial generation override, starting from 0 01:52:06,159 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:52:06,159 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3f3f2beb to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@cdbfcf4 01:52:06,159 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@8e469b7 01:52:06,159 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:52:06,159 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:06,160 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, 0, 1710165126159, None)) 01:52:06,160 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165126158,0,0)) 01:52:06,160 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@21a092de 01:52:06,160 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@cdbfcf4 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7c61bac7 01:52:06,160 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} started 01:52:06,160 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:52:06,160 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:06,160 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:06,160 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:06,160 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: updated peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding on shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding#-286055516] 01:52:06,161 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithShardLeaderNotResponding, persistent : true 01:52:06,161 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithShardLeaderNotResponding with journal batch size 1 01:52:06,161 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-654693386 created and ready for shard:member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:06,161 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding#-286055516] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:06,162 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-2:snapshot-read#-505313036] 01:52:06,162 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:52:06,162 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:52:06,162 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Initializing shard [cars] 01:52:06,162 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding is initialized 01:52:06,162 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 01:52:06,162 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 01:52:06,162 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from null to Follower 01:52:06,162 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:06,162 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithShardLeaderNotResponding , registered listener akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding 01:52:06,162 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-2-shard-cars-testTransactionWithShardLeaderNotResponding from null to Follower 01:52:06,162 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-654693386] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:06,162 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-2:snapshot-read#-505313036] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:06,210 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#1831734487] 01:52:06,210 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: findLocalShard : cars 01:52:06,210 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-119000179] 01:52:06,411 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:52:06,412 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Received ElectionTimeout - switching to Candidate 01:52:06,412 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithShardLeaderNotResponding] 01:52:06,412 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:06,412 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 1 01:52:06,412 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:52:06,412 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Closing 01:52:06,412 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 01:52:06,412 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 01:52:06,412 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from Follower to Candidate 01:52:06,412 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from Follower to Candidate 01:52:06,692 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 01:52:06,692 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:06,692 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 2 01:52:07,12 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 01:52:07,12 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:07,12 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 3 01:52:07,93 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:52:07,93 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:52:07,93 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:52:07,93 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:07,93 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:07,93 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:52:07,93 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:07,93 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:07,93 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:07,322 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 01:52:07,322 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:07,322 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 4 01:52:07,322 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:07,323 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:07,323 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Cluster state: [] 01:52:07,323 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:52:07,323 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:52:07,323 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=null 01:52:07,323 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:52:07,323 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:07,323 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:52:07,329 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:52:07,329 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:52:07,329 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithShardLeaderNotResponding] 01:52:07,329 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:52:07,329 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:52:07,329 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: true, hasLeader : true 01:52:07,329 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: became leader with frontend state for [] 01:52:07,330 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from Candidate to Leader 01:52:07,330 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderPayloadVersion=13] 01:52:07,330 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:07,330 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:52:07,330 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from Candidate to Leader 01:52:07,330 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 01:52:07,331 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Closing 01:52:07,331 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithShardLeaderNotResponding peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:07,331 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding Received follower initial sync status for member-2-shard-cars-testTransactionWithShardLeaderNotResponding status sync done false 01:52:07,331 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithShardLeaderNotResponding, leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderPayloadVersion=13] 01:52:07,331 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:07,331 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:52:07,331 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 01:52:07,331 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithShardLeaderNotResponding updated: matchIndex: -1, nextIndex: 0 01:52:07,331 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:52:07,367 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7b967192: creating a new snapshot member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 01:52:07,367 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} 01:52:07,367 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:07,367 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:07,367 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#1260191167] 01:52:07,367 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, history=0, cookie=0} shard 1 01:52:07,367 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:07,367 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 01:52:07,367 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:52:07,367 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1257389175], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@1db87058 01:52:07,367 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding, primaryVersion=13] 01:52:07,367 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} scheduling timeout in 2 seconds 01:52:07,367 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding, primaryVersion=13] 01:52:07,368 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6fd7ce00, attempting to connect 01:52:07,374 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} is not yet known 01:52:07,374 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, nanosAgo=12960, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:52:07,375 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:52:07,376 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-119000179], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:52:07,376 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-119000179], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:07,376 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@f0272ea instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2a7165d8 01:52:07,376 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:52:07,376 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2a7165d8 01:52:07,376 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1 01:52:07,376 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} 01:52:07,376 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1257389175], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} 01:52:07,376 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:52:07,376 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1257389175], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1247/0x0000000801301400@1b91c9c8 01:52:07,376 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-119000179], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1991410767 nanoseconds 01:52:07,376 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6b206d8e transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1257389175], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=881698394144} 01:52:07,376 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7b967192 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-119000179], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:07,376 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} 01:52:07,376 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@4a5e9a97 01:52:07,376 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@d4a17b2 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1800505a, splicing queue 01:52:07,376 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@d4a17b2 queue spliced 0 messages 01:52:07,376 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-119000179], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 682.8 μs 01:52:07,377 PM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: allocating new open snapshot member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1 01:52:07,377 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithShardLeaderNotResponding_slicer-id-703: slice: identifier: member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0}} 01:52:07,377 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionWithShardLeaderNotResponding_slicer-id-703: Message size: 235 bytes, total slices to send: 1 01:52:07,377 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithShardLeaderNotResponding_slicer-id-703: Message does not need to be sliced - sending original message 01:52:07,378 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1257389175], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0}} 01:52:07,378 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1257389175], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=881698394144, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0}} 01:52:07,378 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0} 01:52:07,378 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 01:52:07,379 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:07,379 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader)_slicer-id-706: Closing 01:52:07,379 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Aborting 0 pending queued transactions 01:52:07,379 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 01:52:07,380 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:07,380 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:07,391 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:07,391 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:07,391 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:07,391 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:07,392 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:52:07,405 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithShardLeaderNotResponding 01:52:07,405 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7b967192: creating a new transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 01:52:07,405 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 01:52:07,405 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:52:07,405 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7b967192 readied transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 01:52:07,405 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1257389175], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@65882e26 01:52:07,405 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:07,405 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-119000179], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:07,406 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:52:07,406 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6b206d8e transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1257389175], modifications=1, protocol=SIMPLE}, enqueuedTicks=881736485908} 01:52:07,406 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-704: slice: identifier: member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1257389175], modifications=1, protocol=SIMPLE}} 01:52:07,406 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-704: Message size: 398 bytes, total slices to send: 1 01:52:07,406 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-704: Message does not need to be sliced - sending original message 01:52:08,154 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:52:08,156 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:52:08,156 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:08,156 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:08,156 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 01:52:08,156 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:52:08,156 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:08,156 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:08,156 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 01:52:08,395 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [7557809652491913287] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:52:08,398 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [7557809652491913287] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:52:09,379 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: running timer on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} 01:52:09,379 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} is empty 01:52:09,379 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: not scheduling timeout on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} 01:52:09,387 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: running timer on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-119000179], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:09,387 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-119000179], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 18040459 nanoseconds 01:52:09,428 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: running timer on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-119000179], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:09,428 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: Connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-119000179], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} timed out entry TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1257389175], modifications=1, protocol=SIMPLE}, enqueuedTicks=881736485908, sessionId=0, txSequence=1} 01:52:09,428 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-119000179], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} timed out 1 tasks 01:52:09,428 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: not scheduling timeout on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-119000179], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:09,428 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=FLUSHED} directCommit completed 01:52:09,428 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=FLUSHED}: initiating purge 01:52:09,428 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=DONE} allocated sequence 1 01:52:09,428 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1257389175]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@702402c9 01:52:09,428 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-119000179], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1999965359 nanoseconds 01:52:09,428 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6b206d8e transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1257389175]}, enqueuedTicks=883759124386} 01:52:09,429 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:52:09,429 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithShardLeaderNotResponding data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:52:09,429 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 01:52:09,429 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 01:52:09,429 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithShardLeaderNotResponding data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:52:09,430 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 01:52:09,430 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 01:52:09,430 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 01:52:09,430 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Closing 01:52:09,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Aborting 0 pending queued transactions 01:52:09,431 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 01:52:09,431 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:09,431 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:10,444 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-815793374557718875] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:52:10,449 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:10,450 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:10,451 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:52:10.452] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:10,459 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:10,459 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:10,461 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:52:10.462] [cluster-test-akka.actor.internal-dispatcher-6] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:10,482 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:52:10,487 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-831919427409769538] 01:52:10,487 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:52:10,489 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:52:10,489 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:52:10,494 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:10,494 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:10,498 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:52:10,498 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:52:10,498 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:52:10,502 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:52:10,507 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-1191921172140774911] 01:52:10,507 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:52:10,508 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:10,508 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:52:10,515 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:10,515 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:10,524 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:52:10,530 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-6811320029992040581] 01:52:10,530 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:52:10,531 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:10,531 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:52:10,533 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:10,533 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:10,540 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:10,541 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:10,542 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 01:52:10,548 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:52:10,553 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#507049234] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:10,557 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:52:10,560 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:52:10,564 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 01:52:10,564 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:10,564 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:52:10,564 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:10,566 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: no initial generation override, starting from 0 01:52:10,566 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:52:10,566 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:52:10,566 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@746ebb54 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@13dd2ccb 01:52:10,566 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@56dfa0d4 01:52:10,567 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1710165130566, None)) 01:52:10,567 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165130565,0,0)) 01:52:10,568 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@13dd2ccb to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6622be62 01:52:10,568 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} started 01:52:10,569 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:10,569 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:52:10,570 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:10,572 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5950158a 01:52:10,572 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:52:10,572 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:10,572 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 01:52:10,573 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:10,573 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:10,573 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: updated peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008] 01:52:10,573 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 01:52:10,574 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:10,574 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 01:52:10,574 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-1110566741] 01:52:10,574 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-1141866177 created and ready for shard:member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:10,574 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-1141866177] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:10,574 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-1110566741] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:10,576 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:52:10,576 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:52:10,576 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:52:10,576 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:52:10,576 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 01:52:10,576 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 01:52:10,576 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 01:52:10,576 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#1653348113] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:10,576 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 01:52:10,579 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:52:10,580 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 01:52:10,581 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: no initial generation override, starting from 0 01:52:10,581 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:52:10,581 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@e5fddb6 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@22946ff5 01:52:10,581 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 01:52:10,581 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:10,581 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3b532baa 01:52:10,581 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:52:10,581 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:10,582 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1710165130582, None)) 01:52:10,582 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165130581,0,0)) 01:52:10,582 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@22946ff5 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@74a4c473 01:52:10,582 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} started 01:52:10,583 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7134a15d 01:52:10,583 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:52:10,583 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:10,583 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:10,583 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:10,583 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: updated peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1402098955] 01:52:10,583 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Shard created : member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 01:52:10,583 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1402098955] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:10,583 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-1965455223 created and ready for shard:member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:10,584 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-1965455223] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:10,584 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-2:snapshot-read#-791325121] 01:52:10,584 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 01:52:10,584 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-2:snapshot-read#-791325121] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:10,585 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:52:10,585 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:52:10,585 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:52:10,585 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:52:10,586 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:10,586 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 01:52:10,586 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 01:52:10,586 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 01:52:10,586 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 01:52:10,586 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 01:52:10,633 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#507049234] 01:52:10,633 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 01:52:10,633 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008] 01:52:10,858 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:52:10,858 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 01:52:10,858 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:52:10,859 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:10,859 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 1 01:52:10,859 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:52:10,859 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:52:10,859 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:52:10,859 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:52:10,859 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 01:52:10,859 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 01:52:11,128 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:11,128 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,128 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 2 01:52:11,130 PM [cluster-test-shard-dispatcher-16] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:52:11,377 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:11,377 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,378 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 3 01:52:11,509 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:52:11,509 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,509 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,509 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:52:11,509 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:52:11,509 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,509 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,509 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:52:11,509 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,514 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:52:11,514 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,514 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,514 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:52:11,514 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,514 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,551 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:52:11,618 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:11,618 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,618 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 4 01:52:11,618 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,619 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:11,619 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:52:11,619 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:52:11,619 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:52:11,619 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=null 01:52:11,619 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:52:11,619 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,619 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:52:11,620 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:52:11,620 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:52:11,620 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:52:11,620 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:52:11,620 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:52:11,620 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 01:52:11,621 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [] 01:52:11,621 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Leader 01:52:11,621 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 01:52:11,621 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:11,621 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:52:11,621 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Leader 01:52:11,621 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 01:52:11,621 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 01:52:11,621 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,622 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 01:52:11,622 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 01:52:11,622 PM [cluster-test-shard-dispatcher-16] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:11,622 PM [cluster-test-shard-dispatcher-16] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:52:11,622 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 01:52:11,622 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: -1, nextIndex: 0 01:52:11,622 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:52:11,639 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1b14c609: creating a new transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:52:11,639 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} 01:52:11,639 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:11,639 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:11,639 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#1653348113] 01:52:11,639 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} shard 1 01:52:11,639 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:52:11,639 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:52:11,639 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1b14c609 readied transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 01:52:11,639 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], modifications=2, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@34b964f 01:52:11,639 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} scheduling timeout in 2 seconds 01:52:11,639 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:11,639 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 01:52:11,639 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 01:52:11,639 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@1eb9dc34, attempting to connect 01:52:11,640 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} is not yet known 01:52:11,641 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, nanosAgo=10411, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:52:11,641 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:52:11,641 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:52:11,641 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:11,641 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@380aa2dc instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@39cf3758 01:52:11,641 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:52:11,641 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@39cf3758 01:52:11,641 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:52:11,641 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} 01:52:11,641 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], modifications=2, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} 01:52:11,641 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:52:11,641 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], modifications=2, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2277/0x00000008016c5c10@7b6335cb 01:52:11,641 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 01:52:11,641 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@62fe312d transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], modifications=2, protocol=SIMPLE}, enqueuedTicks=885970072171} 01:52:11,642 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-709: slice: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], modifications=2, protocol=SIMPLE}} 01:52:11,642 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-709: Message size: 413 bytes, total slices to send: 1 01:52:11,642 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-709: Message does not need to be sliced - sending original message 01:52:11,642 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1b14c609 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:11,642 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} 01:52:11,642 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@6cd155ce 01:52:11,642 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@14ed17b2 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@31e02f51, splicing queue 01:52:11,642 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@14ed17b2 queue spliced 0 messages 01:52:11,642 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 509.1 μs 01:52:11,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: allocating new open transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:52:11,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: readying transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:52:11,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 ready 01:52:11,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transitioned member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 to ready 01:52:11,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 initiating direct canCommit 01:52:11,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Validating transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:52:11,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 validated 01:52:11,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:52:11,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 initiating direct preCommit 01:52:11,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Preparing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:52:11,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 candidate ready 01:52:11,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1: canCommit - candidate rootPath: / 01:52:11,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1: canCommit - messages: [] 01:52:11,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 01:52:11,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 prepared 01:52:11,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:52:11,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 initiating direct commit 01:52:11,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Starting commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:52:11,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 182 01:52:11,644 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}] 01:52:11,644 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}] 01:52:11,644 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:52:11,644 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:52:11,644 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,644 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}]]] 01:52:11,644 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 submitted to persistence 01:52:11,644 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}] 01:52:11,645 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}]]] 01:52:11,645 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 01:52:11,645 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:52:11,645 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log Simple{size=182} 01:52:11,645 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}] 01:52:11,645 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 1 01:52:11,645 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:52:11,645 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}] 01:52:11,645 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 01:52:11,645 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 0, nextIndex: 1 01:52:11,645 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:52:11,645 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 0 01:52:11,645 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 01:52:11,645 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 0 data Simple{size=182} 01:52:11,645 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Resuming commit of transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:52:11,645 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1d975107}, purged=MutableUnsignedLongSet{size=0}} 01:52:11,645 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 01:52:11,645 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:52:11,645 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1: commit - successfulFromPrevious: [] 01:52:11,645 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:52:11,646 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#17664564] with name member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 01:52:11,646 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#17664564] 01:52:11,646 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 01:52:11,646 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:52:11,646 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 51.53 μs 01:52:11,646 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=0 01:52:11,646 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#17664564] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:11,646 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 0 01:52:11,646 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:52:11,646 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 0 01:52:11,646 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 01:52:11,646 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 0 data Simple{size=182} 01:52:11,647 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], modifications=2, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0}} 01:52:11,647 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], modifications=2, protocol=SIMPLE}, enqueuedTicks=885970072171, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0}} 01:52:11,647 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying foreign transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:52:11,647 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], modifications=2, protocol=SIMPLE} completed with TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0} 01:52:11,647 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} directCommit completed 01:52:11,647 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 01:52:11,647 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:52:11,647 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-740505897]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@30365446 01:52:11,647 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} 01:52:11,647 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SEALED}: initiating purge 01:52:11,647 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} allocated sequence 1 01:52:11,647 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-740505897]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@562f18ab 01:52:11,647 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:11,647 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendMetadata - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@6e091a5c}, purged=MutableUnsignedLongSet{size=0}} 01:52:11,647 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@62fe312d transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-740505897]}, enqueuedTicks=885977659519} 01:52:11,647 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 01:52:11,647 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:52:11,647 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#933592620] with name member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 01:52:11,647 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#933592620] 01:52:11,647 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=0 01:52:11,647 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 01:52:11,647 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#933592620] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:11,647 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 70.21 μs 01:52:11,648 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: purging transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:52:11,648 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 01:52:11,648 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 01:52:11,648 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:52:11,648 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testSingleShardTransactionsWithLeaderChanges 01:52:11,648 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:52:11,648 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 01:52:11,649 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,649 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-711: Closing 01:52:11,649 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 01:52:11,649 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:52:11,649 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:11,649 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:11,654 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:11,654 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:11,654 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:11,654 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:11,655 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:52:11,656 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,656 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 01:52:11,656 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Re-initializing to Follower with previous leaderId member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,656 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:52:11,656 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : true 01:52:11,656 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:52:11,670 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 01:52:11,720 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#1653348113] 01:52:11,720 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 01:52:11,721 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1402098955] 01:52:11,721 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,771 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,811 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 01:52:11,811 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 01:52:11,811 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 01:52:11,811 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 01:52:11,811 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:52:11,811 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,811 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 5 01:52:11,811 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,811 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:52:11,811 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:52:11,811 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:52:11,811 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:52:11,811 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 01:52:11,812 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 01:52:11,812 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=-1] 01:52:11,812 PM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:11,812 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 01:52:11,812 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:52:11,812 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@62fe312d superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@36fbb766, splicing queue 01:52:11,812 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 01:52:11,812 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@62fe312d queue spliced 1 messages 01:52:11,812 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:11,812 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: refreshing backend for shard 1 01:52:11,812 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:52:11,812 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:11,812 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:11,812 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#1653348113] 01:52:11,813 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:11,813 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Candidate, leaderId=null, activeMember=true] 01:52:11,828 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:52:11,833 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [3789429996515803130] 01:52:11,833 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:52:11,834 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:52:11,834 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:52:11,841 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:11,841 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:11,843 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:11,845 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:11,846 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 01:52:11,865 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@f0272ea aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} 01:52:11,865 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:52:11,865 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1257389175], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@53c2519 01:52:11,865 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} has a successor, not scheduling timer 01:52:11,865 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SEALED} allocated sequence 1 01:52:11,865 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1257389175], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1240/0x0000000801303148@78cee517 01:52:11,865 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6b206d8e transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1257389175], modifications=0, protocol=ABORT}, enqueuedTicks=886196149772} 01:52:11,865 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-704: slice: identifier: member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=3, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1257389175], modifications=0, protocol=ABORT}} 01:52:11,865 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#207195453] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:11,865 PM [Cleaner-1] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-704: Message size: 293 bytes, total slices to send: 1 01:52:11,865 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-704: Message does not need to be sliced - sending original message 01:52:11,865 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0} 01:52:11,865 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3cba15e0 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-1, state=OPEN} 01:52:11,865 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-264245909]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@634e3ecc 01:52:11,865 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 01:52:11,865 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:52:11,866 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5f9cafb6 aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} 01:52:11,866 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:52:11,866 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-264245909], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@44b898f4 01:52:11,866 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} 01:52:11,866 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3cba15e0 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 01:52:11,866 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-264245909]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1214/0x00000008012fa848@215980c6 01:52:11,866 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 01:52:11,866 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#95236903], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:52:11,867 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:52:11,867 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: recovered identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:52:11,868 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} 01:52:11,869 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4f56b548 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2b3791cf 01:52:11,869 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@44768e71 01:52:11,869 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1710165131869, None)) 01:52:11,869 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165131868,0,0)) 01:52:11,869 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2b3791cf to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3aff9085 01:52:11,869 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} started 01:52:11,877 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 01:52:11,877 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,877 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1e3d470d 01:52:11,877 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:52:11,877 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,878 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 01:52:11,878 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-39899300] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:11,878 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#238124737 created and ready for shard:member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,878 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#238124737] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:11,878 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-1006679473] 01:52:11,878 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 01:52:11,878 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-1006679473] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:11,879 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:52:11,879 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,879 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:52:11,879 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,879 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:52:11,879 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,879 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:52:11,879 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:11,879 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 01:52:11,879 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received ReplicatedLogEntry for recovery: index: 0, size: 182 01:52:11,879 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 01:52:11,879 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received apply journal entries for recovery, applying to state: 0 to 0 01:52:11,879 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: starting log recovery batch with max size 1 01:52:11,879 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@32fbee1e}, purged=MutableUnsignedLongSet{size=0}} 01:52:11,879 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 01:52:11,879 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:52:11,880 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 01:52:11,880 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 01:52:11,880 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received ReplicatedLogEntry for recovery: index: 1, size: 74 01:52:11,880 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:52:11,880 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed in 425.9 μs - Switching actor to Follower - last log index = 1, last log term = 4, snapshot index = -1, snapshot term = -1, journal size = 2 01:52:11,880 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 01:52:11,880 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 01:52:11,880 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:52:11,880 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:52:11,880 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 01:52:11,880 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 01:52:11,880 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 01:52:11,920 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#1653348113] 01:52:11,920 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 01:52:11,920 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1402098955] 01:52:12,0 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:12,0 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:12,0 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 6 01:52:12,0 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:12,30 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received ShardNotInitializedTimeout message for shard cars 01:52:12,30 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Returning NoShardLeaderException for shard cars 01:52:12,31 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:52:12,31 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:12,31 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:12,31 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:12,131 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:12,131 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:12,131 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 7 01:52:12,131 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:12,330 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:12,330 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:12,330 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 8 01:52:12,330 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:12,485 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(akka://cluster-test@127.0.0.1:2558, Removed) 01:52:12,485 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 01:52:12,485 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:52:12,485 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:12,486 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 5 01:52:12,486 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 5 01:52:12,486 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:52:12,486 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:52:12,486 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:52:12,486 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 01:52:12,486 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 01:52:12,551 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:12,551 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:12,551 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 9 01:52:12,551 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:12,660 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [-831919427409769538] but current is [3789429996515803130]. Reason: ActorSystem terminated 01:52:12,672 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-831919427409769538] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:52:12,687 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4], my term is 5 01:52:12,687 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 6 in "RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than Candidate's term 5 - switching to Follower 01:52:12,687 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=null 01:52:12,687 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 1, lastTerm: 4 01:52:12,687 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=6, voteGranted=false] 01:52:12,687 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 6 01:52:12,687 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:52:12,687 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:52:12,687 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 01:52:12,687 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] LocalActorRefProvider$$anon$2 - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] wrapped in [akka.actor.ActorSelectionMessage] to Actor[akka://cluster-test/] was dropped. Unknown system with UID [-1191921172140774911]. This system with UID [3789429996515803130] was probably restarted. Messages will be accepted when new handshake has been completed.. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:12,688 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 01:52:12,688 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#-658011439]] intended for system uid [-831919427409769538], self uid is [3789429996515803130] 01:52:12,688 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:12,688 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:52:12,688 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:52:12,688 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 7 in "RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 6 - updating term 01:52:12,688 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=null 01:52:12,688 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 7, votedFor: null, lastIndex: 1, lastTerm: 4 01:52:12,688 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=7, voteGranted=false] 01:52:12,688 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=false], my term is 9 01:52:12,688 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2558, Leaving)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 01:52:12,688 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=false], current voteCount: 1 01:52:12,688 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:12,688 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:52:12,688 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:52:12,688 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 8 in "RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 7 - updating term 01:52:12,688 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=null 01:52:12,688 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 8, votedFor: null, lastIndex: 1, lastTerm: 4 01:52:12,688 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=8, voteGranted=false] 01:52:12,689 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:12,689 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:52:12,689 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:52:12,689 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 9 in "RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 8 - updating term 01:52:12,689 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=null 01:52:12,689 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 9, votedFor: null, lastIndex: 1, lastTerm: 4 01:52:12,689 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=9, voteGranted=false] 01:52:12,689 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking unreachable node [akka://cluster-test@127.0.0.1:2558] as [Down] 01:52:12,689 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=7, voteGranted=false], my term is 9 01:52:12,689 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=7, voteGranted=false], current voteCount: 1 01:52:12,689 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=8, voteGranted=false], my term is 9 01:52:12,689 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=8, voteGranted=false], current voteCount: 1 01:52:12,689 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:52:12,690 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=9, voteGranted=false], my term is 9 01:52:12,690 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=9, voteGranted=false], current voteCount: 1 01:52:12,694 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] LocalActorRefProvider$$anon$2 - Message [akka.cluster.GossipEnvelope] wrapped in [akka.actor.ActorSelectionMessage] to Actor[akka://cluster-test/] was dropped. Unknown system with UID [-6811320029992040581]. This system with UID [3789429996515803130] was probably restarted. Messages will be accepted when new handshake has been completed.. [6] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:12,721 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:12,721 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:12,721 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 10 01:52:12,721 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:12,722 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:12,722 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:52:12,722 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:52:12,722 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 10 in "RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 9 - updating term 01:52:12,722 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=null 01:52:12,722 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 10, votedFor: null, lastIndex: 1, lastTerm: 4 01:52:12,722 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=10, voteGranted=false] 01:52:12,723 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=10, voteGranted=false], my term is 10 01:52:12,723 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=10, voteGranted=false], current voteCount: 1 01:52:12,857 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2558, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 01:52:12,931 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:12,931 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=11, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:12,931 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 11 01:52:12,931 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=11, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:12,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:12,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:52:12,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=11, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:52:12,932 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 11 in "RequestVote [term=11, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 10 - updating term 01:52:12,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=11, votedFor=null 01:52:12,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=11, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 11, votedFor: null, lastIndex: 1, lastTerm: 4 01:52:12,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=11, voteGranted=false] 01:52:12,933 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=11, voteGranted=false], my term is 11 01:52:12,933 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=11, voteGranted=false], current voteCount: 1 01:52:13,51 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:13,51 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:13,51 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:13,51 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#1653348113] 01:52:13,51 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:13,51 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Candidate, leaderId=null, activeMember=true] 01:52:13,100 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:13,100 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=12, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:13,100 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 12 01:52:13,100 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=12, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:13,102 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:13,102 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:52:13,102 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=12, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:52:13,102 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 12 in "RequestVote [term=12, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 11 - updating term 01:52:13,102 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=12, votedFor=null 01:52:13,102 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=12, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 12, votedFor: null, lastIndex: 1, lastTerm: 4 01:52:13,102 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=12, voteGranted=false] 01:52:13,103 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=12, voteGranted=false], my term is 12 01:52:13,103 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=12, voteGranted=false], current voteCount: 1 01:52:13,225 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(akka://cluster-test@127.0.0.1:2558, Removed) 01:52:13,225 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 01:52:13,225 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:52:13,225 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=13, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:13,225 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 13 01:52:13,226 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 13 01:52:13,226 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:52:13,226 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:52:13,226 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:52:13,226 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 01:52:13,226 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 01:52:13,271 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received ShardNotInitializedTimeout message for shard cars 01:52:13,271 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Returning NoShardLeaderException for shard cars 01:52:13,271 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:13,271 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=13, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:13,271 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 13 01:52:13,271 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:52:13,271 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=13, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:13,271 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:13,272 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:13,272 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:13,272 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=13, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4], my term is 13 01:52:13,272 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=13, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 13, votedFor: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastIndex: 1, lastTerm: 4 01:52:13,272 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=13, voteGranted=false] 01:52:13,273 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=13, voteGranted=false], my term is 13 01:52:13,273 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=13, voteGranted=false], current voteCount: 1 01:52:13,481 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:13,481 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=14, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:13,481 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 14 01:52:13,481 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=14, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:13,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=14, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4], my term is 13 01:52:13,482 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 14 in "RequestVote [term=14, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than Candidate's term 13 - switching to Follower 01:52:13,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=14, votedFor=null 01:52:13,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=14, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 14, votedFor: null, lastIndex: 1, lastTerm: 4 01:52:13,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=14, voteGranted=false] 01:52:13,482 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 14 01:52:13,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:52:13,482 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:52:13,482 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 01:52:13,483 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 01:52:13,483 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=14, voteGranted=false], my term is 14 01:52:13,483 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=14, voteGranted=false], current voteCount: 1 01:52:13,592 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is removing unreachable node [akka://cluster-test@127.0.0.1:2558] 01:52:13,593 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [-831919427409769538] but current is [3789429996515803130]. Reason: Cluster member removed, previous status [Down] 01:52:13,651 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: running timer on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} 01:52:13,651 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} is empty 01:52:13,651 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: not scheduling timeout on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} 01:52:13,661 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: running timer on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:13,661 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} is empty 01:52:13,661 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: not scheduling timeout on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:13,690 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:13,690 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=15, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:13,690 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 15 01:52:13,690 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=15, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:13,691 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:13,691 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:52:13,691 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=15, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:52:13,692 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 15 in "RequestVote [term=15, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 14 - updating term 01:52:13,692 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=15, votedFor=null 01:52:13,692 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=15, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 15, votedFor: null, lastIndex: 1, lastTerm: 4 01:52:13,692 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=15, voteGranted=false] 01:52:13,692 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=15, voteGranted=false], my term is 15 01:52:13,692 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=15, voteGranted=false], current voteCount: 1 01:52:13,830 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: running timer on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:13,831 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 01:52:13,861 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:13,861 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=16, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:13,861 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 16 01:52:13,861 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=16, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:13,862 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:13,862 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:52:13,862 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=16, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:52:13,862 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 16 in "RequestVote [term=16, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 15 - updating term 01:52:13,862 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=16, votedFor=null 01:52:13,862 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=16, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 16, votedFor: null, lastIndex: 1, lastTerm: 4 01:52:13,862 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=16, voteGranted=false] 01:52:13,863 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=16, voteGranted=false], my term is 16 01:52:13,863 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=16, voteGranted=false], current voteCount: 1 01:52:13,876 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2558, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 01:52:13,938 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [-831919427409769538] but current is [3789429996515803130]. Reason: Cluster member removed, previous status [Down] 01:52:13,938 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberRemoved: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:52:14,5 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(akka://cluster-test@127.0.0.1:2558, Removed) 01:52:14,5 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 01:52:14,6 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:52:14,6 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=17, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:14,6 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 17 01:52:14,6 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 17 01:52:14,6 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:52:14,6 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:52:14,6 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:52:14,6 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 01:52:14,6 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 01:52:14,70 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:14,70 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=17, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:14,70 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 17 01:52:14,70 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=17, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:14,71 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=17, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4], my term is 17 01:52:14,71 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=17, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 17, votedFor: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastIndex: 1, lastTerm: 4 01:52:14,71 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=17, voteGranted=false] 01:52:14,72 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=17, voteGranted=false], my term is 17 01:52:14,72 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=17, voteGranted=false], current voteCount: 1 01:52:14,280 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:14,280 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=18, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:14,280 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 18 01:52:14,280 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=18, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:14,281 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=18, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4], my term is 17 01:52:14,281 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 18 in "RequestVote [term=18, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than Candidate's term 17 - switching to Follower 01:52:14,281 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=18, votedFor=null 01:52:14,281 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=18, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 18, votedFor: null, lastIndex: 1, lastTerm: 4 01:52:14,281 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=18, voteGranted=false] 01:52:14,281 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 18 01:52:14,281 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:52:14,281 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:52:14,281 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 01:52:14,282 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 01:52:14,282 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=18, voteGranted=false], my term is 18 01:52:14,282 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=18, voteGranted=false], current voteCount: 1 01:52:14,290 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:14,290 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:14,290 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:14,290 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#1653348113] 01:52:14,290 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:14,290 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Candidate, leaderId=null, activeMember=true] 01:52:14,410 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:14,410 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=19, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:14,411 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 19 01:52:14,411 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=19, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:14,412 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:14,412 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:52:14,412 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=19, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:52:14,412 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 19 in "RequestVote [term=19, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 18 - updating term 01:52:14,412 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=19, votedFor=null 01:52:14,412 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=19, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 19, votedFor: null, lastIndex: 1, lastTerm: 4 01:52:14,412 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=19, voteGranted=false] 01:52:14,413 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=19, voteGranted=false], my term is 19 01:52:14,413 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=19, voteGranted=false], current voteCount: 1 01:52:14,510 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received ShardNotInitializedTimeout message for shard cars 01:52:14,510 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Returning NoShardLeaderException for shard cars 01:52:14,511 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:52:14,511 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:14,511 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:14,511 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:14,580 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:14,580 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=20, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:14,580 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 20 01:52:14,580 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=20, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:14,581 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:14,581 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:52:14,582 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=20, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:52:14,582 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 20 in "RequestVote [term=20, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 19 - updating term 01:52:14,582 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=20, votedFor=null 01:52:14,582 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=20, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 20, votedFor: null, lastIndex: 1, lastTerm: 4 01:52:14,582 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=20, voteGranted=false] 01:52:14,582 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=20, voteGranted=false], my term is 20 01:52:14,582 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=20, voteGranted=false], current voteCount: 1 01:52:14,780 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:14,780 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=21, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:14,780 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 21 01:52:14,780 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=21, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:14,782 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:14,782 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:52:14,782 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=21, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:52:14,782 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 21 in "RequestVote [term=21, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 20 - updating term 01:52:14,782 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=21, votedFor=null 01:52:14,782 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=21, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 21, votedFor: null, lastIndex: 1, lastTerm: 4 01:52:14,782 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=21, voteGranted=false] 01:52:14,783 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=21, voteGranted=false], my term is 21 01:52:14,783 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=21, voteGranted=false], current voteCount: 1 01:52:14,825 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(akka://cluster-test@127.0.0.1:2558, Removed) 01:52:14,825 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 01:52:14,825 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:52:14,825 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=22, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:14,825 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 22 01:52:14,825 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 22 01:52:14,825 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:52:14,825 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:52:14,825 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:52:14,825 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 01:52:14,825 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 01:52:14,896 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING, roles [member-1, dc-default], version [0.0.0] 01:52:14,899 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Welcome from [akka://cluster-test@127.0.0.1:2559] 01:52:14,900 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:52:14,900 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:14,900 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:14,900 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:52:14,900 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:14,900 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:14,900 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:14,990 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:14,990 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=22, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:14,990 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 22 01:52:14,990 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=22, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:14,991 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=22, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4], my term is 22 01:52:14,991 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=22, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 22, votedFor: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastIndex: 1, lastTerm: 4 01:52:14,991 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=22, voteGranted=false] 01:52:14,992 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=22, voteGranted=false], my term is 22 01:52:14,992 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=22, voteGranted=false], current voteCount: 1 01:52:15,131 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:15,131 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=23, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:15,131 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 23 01:52:15,131 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=23, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:15,131 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=23, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4], my term is 22 01:52:15,132 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 23 in "RequestVote [term=23, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than Candidate's term 22 - switching to Follower 01:52:15,132 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=23, votedFor=null 01:52:15,132 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=23, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 23, votedFor: null, lastIndex: 1, lastTerm: 4 01:52:15,132 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=23, voteGranted=false] 01:52:15,132 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 23 01:52:15,132 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:52:15,132 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:52:15,132 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 01:52:15,132 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 01:52:15,132 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=23, voteGranted=false], my term is 23 01:52:15,132 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=23, voteGranted=false], current voteCount: 1 01:52:15,350 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:15,350 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=24, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:15,350 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 24 01:52:15,350 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=24, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:15,352 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:15,352 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:52:15,352 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=24, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:52:15,352 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 24 in "RequestVote [term=24, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 23 - updating term 01:52:15,352 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=24, votedFor=null 01:52:15,352 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=24, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 24, votedFor: null, lastIndex: 1, lastTerm: 4 01:52:15,352 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=24, voteGranted=false] 01:52:15,353 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=24, voteGranted=false], my term is 24 01:52:15,353 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=24, voteGranted=false], current voteCount: 1 01:52:15,530 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:15,530 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:15,530 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=25, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:15,530 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:15,530 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:15,530 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 25 01:52:15,530 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#1653348113] 01:52:15,531 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:15,531 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=25, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:15,531 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Candidate, leaderId=null, activeMember=true] 01:52:15,532 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:15,532 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:52:15,532 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=25, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:52:15,532 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 25 in "RequestVote [term=25, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 24 - updating term 01:52:15,532 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=25, votedFor=null 01:52:15,532 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=25, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 25, votedFor: null, lastIndex: 1, lastTerm: 4 01:52:15,532 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=25, voteGranted=false] 01:52:15,533 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=25, voteGranted=false], my term is 25 01:52:15,533 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=25, voteGranted=false], current voteCount: 1 01:52:15,632 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:52:15,690 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 01:52:15,690 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=26, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:15,690 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 26 01:52:15,690 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=26, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:15,691 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:15,691 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 01:52:15,691 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=26, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 01:52:15,691 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 26 in "RequestVote [term=26, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 25 - updating term 01:52:15,691 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=26, votedFor=null 01:52:15,691 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=26, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 26, votedFor: null, lastIndex: 1, lastTerm: 4 01:52:15,691 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=26, voteGranted=false] 01:52:15,691 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=26, voteGranted=false], my term is 26 01:52:15,692 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=26, voteGranted=false], current voteCount: 1 01:52:15,725 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Joining), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Joining) 01:52:15,725 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 01:52:15,725 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:52:15,725 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=27, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:15,725 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 27 01:52:15,725 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=27, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:15,725 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 27 01:52:15,725 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:52:15,725 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:52:15,725 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:52:15,725 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 01:52:15,726 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 01:52:15,727 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=27, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4], my term is 26 01:52:15,727 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 27 in "RequestVote [term=27, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than Candidate's term 26 - switching to Follower 01:52:15,727 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=27, votedFor=null 01:52:15,727 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=27, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 27, votedFor: null, lastIndex: 0, lastTerm: 4 01:52:15,727 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=27, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:15,727 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=27, voteGranted=true] 01:52:15,727 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 27 01:52:15,727 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:52:15,727 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:52:15,727 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 01:52:15,727 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 01:52:15,727 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized - notifying 1 OnShardInitialized callbacks 01:52:15,728 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=27, voteGranted=true], my term is 27 01:52:15,728 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=27, voteGranted=true], current voteCount: 1 01:52:15,728 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): LastApplied index 0 is behind last index 1 - switching to PreLeader 01:52:15,728 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Election: Leader has following peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:52:15,728 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:52:15,728 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 27 01:52:15,728 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 01:52:15,728 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges] 01:52:15,728 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=2, term=27, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] 01:52:15,728 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=27, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] 01:52:15,728 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Replicate message: identifier: null, logIndex: 2, payload: class org.opendaylight.controller.cluster.raft.persisted.NoopPayload, isSendImmediate: true 01:52:15,729 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:52:15,729 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to PreLeader 01:52:15,729 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 01:52:15,729 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:15,729 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:52:15,729 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to PreLeader 01:52:15,729 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=2, term=27, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] 01:52:15,730 PM [cluster-test-shard-dispatcher-16] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 01:52:15,730 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:15,730 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 01:52:15,730 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 01:52:15,730 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 01:52:15,730 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized - notifying 1 OnShardInitialized callbacks 01:52:15,730 PM [cluster-test-shard-dispatcher-16] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:15,730 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:52:15,731 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 0, nextIndex: 1 01:52:15,731 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:52:15,731 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:15,731 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}], SimpleReplicatedLogEntry [index=2, term=27, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e]]] 01:52:15,731 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:15,733 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}], SimpleReplicatedLogEntry [index=2, term=27, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e]]] 01:52:15,733 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 2 01:52:15,733 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:52:15,733 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 01:52:15,733 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 01:52:15,733 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e 01:52:15,733 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=27, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] 01:52:15,733 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 3 01:52:15,733 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=27, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=2, logLastTerm=27, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:52:15,733 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 2, nextIndex: 3 01:52:15,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 4 does not match the current term 27 01:52:15,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 01:52:15,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Setting last applied to 1 01:52:15,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 01:52:15,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 01:52:15,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: replication of member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 has no callback 01:52:15,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 01:52:15,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:52:15,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Setting last applied to 2 01:52:15,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 1 to 2 01:52:15,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e 01:52:15,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 1 for term 27 01:52:15,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:52:15,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Received ApplyState [identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}]] - lastApplied: 2, lastIndex: 2 01:52:15,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:52:15,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 01:52:15,734 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 27 01:52:15,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader)_slicer-id-714: Closing 01:52:15,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 01:52:15,734 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=2 01:52:15,734 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from PreLeader to Leader 01:52:15,734 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 01:52:15,734 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=2, term=27, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e] 01:52:15,735 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from PreLeader to Leader 01:52:15,735 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 01:52:15,735 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 2 01:52:15,735 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 01:52:15,735 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 1 01:52:15,735 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 01:52:15,735 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 01:52:15,735 PM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 01:52:15,735 PM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:52:15,735 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 2 01:52:15,735 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 1 to 2 01:52:15,735 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@3b111b2e 01:52:15,735 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 1 for term 27 01:52:15,735 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=2 01:52:15,746 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:15,746 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1b14c609: creating a new transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:52:15,746 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:52:15,746 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:52:15,746 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1b14c609 readied transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 01:52:15,746 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@61041212 01:52:15,746 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} has a successor, not scheduling timer 01:52:15,750 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received ShardNotInitializedTimeout message for shard cars 01:52:15,750 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Returning NoShardLeaderException for shard cars 01:52:15,751 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:52:15,751 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:15,751 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:15,751 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:15,850 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: running timer on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:15,851 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 01:52:15,926 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:52:15,926 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:15,926 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:15,926 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 01:52:16,634 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:52:16,634 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:16,634 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:16,634 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 01:52:16,634 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:16,770 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:16,770 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:16,770 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:16,770 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#1653348113] 01:52:16,770 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:16,770 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 01:52:16,771 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 01:52:16,771 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@13c33fcd, attempting to connect 01:52:16,773 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:52:16,773 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-39899300], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:52:16,773 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-39899300], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:16,773 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@39cf3758 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6595eea5 01:52:16,773 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} previous state DONE 01:52:16,773 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} previous state FLUSHED 01:52:16,773 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6595eea5 01:52:16,774 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:52:16,774 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} 01:52:16,774 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Replaying queued request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-740505897]} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} 01:52:16,774 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE}: initiating purge 01:52:16,774 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} allocated sequence 0 01:52:16,774 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@1c14e35c 01:52:16,774 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-39899300], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 01:52:16,774 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2b85ecae transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897]}, enqueuedTicks=885977659519} 01:52:16,774 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6595eea5 01:52:16,774 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:52:16,774 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=OPEN} 01:52:16,774 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], modifications=1, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=OPEN} 01:52:16,774 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:52:16,774 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2277/0x00000008016c5c10@6e1103aa 01:52:16,774 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2b85ecae transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], modifications=1, protocol=SIMPLE}, enqueuedTicks=890077376455} 01:52:16,774 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-709: slice: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, message: Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], modifications=1, protocol=SIMPLE}} 01:52:16,774 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-709: Message size: 459 bytes, total slices to send: 1 01:52:16,774 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-709: Message does not need to be sliced - sending original message 01:52:16,775 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1b14c609 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-39899300], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:16,775 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} 01:52:16,775 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} 01:52:16,775 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@2def8a31 01:52:16,775 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3bff1d33 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@180edf70, splicing queue 01:52:16,775 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3bff1d33 queue spliced 0 messages 01:52:16,775 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-39899300], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.435 ms 01:52:16,778 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: purging transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:52:16,778 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=3, term=27, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 01:52:16,778 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=27, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 01:52:16,778 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:52:16,778 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: allocating new open transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: readying transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 ready 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transitioned member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 to ready 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 initiating direct canCommit 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Validating transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 validated 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 initiating direct preCommit 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Preparing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 candidate ready 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1: canCommit - candidate rootPath: / 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1: canCommit - messages: [] 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 prepared 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 initiating direct commit 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Starting commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=4, term=27, payload=Simple{size=241}] 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=27, payload=Simple{size=241}] 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 01:52:16,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:16,780 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=2, prevLogTerm=27, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=27, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}], SimpleReplicatedLogEntry [index=4, term=27, payload=Simple{size=241}]]] 01:52:16,780 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 submitted to persistence 01:52:16,780 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=3, term=27, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 01:52:16,780 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=4, term=27, payload=Simple{size=241}] 01:52:16,780 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=2, prevLogTerm=27, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=27, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}], SimpleReplicatedLogEntry [index=4, term=27, payload=Simple{size=241}]]] 01:52:16,780 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 2 01:52:16,780 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 01:52:16,780 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 01:52:16,780 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=27, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 01:52:16,780 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log Simple{size=241} 01:52:16,780 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=27, payload=Simple{size=241}] 01:52:16,780 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 3 01:52:16,781 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=27, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=4, logLastTerm=27, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:52:16,781 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=3, term=27, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 01:52:16,781 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=4, term=27, payload=Simple{size=241}] 01:52:16,781 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 4, nextIndex: 5 01:52:16,781 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 2 01:52:16,781 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 3 01:52:16,781 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 2 to 3 01:52:16,781 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 01:52:16,781 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: replication of member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2287/0x00000008016da230@558d0ddb 01:52:16,781 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: finished purging inherited transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:52:16,781 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 01:52:16,781 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:52:16,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 4 01:52:16,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 3 to 4 01:52:16,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 4 data Simple{size=241} 01:52:16,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Resuming commit of transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:52:16,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 01:52:16,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:52:16,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1: commit - successfulFromPrevious: [] 01:52:16,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 01:52:16,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#2095567870] with name member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 01:52:16,782 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#2095567870] 01:52:16,782 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 01:52:16,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 3 for term 27 01:52:16,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:52:16,782 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#2095567870] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:16,782 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 124.6 μs 01:52:16,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=4 01:52:16,789 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897]} with Envelope{sessionId=1, txSequence=0, message=TransactionPurgeResponse{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0}} 01:52:16,789 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897]}, enqueuedTicks=885977659519, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionPurgeResponse{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0}} 01:52:16,789 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE}: purge completed 01:52:16,789 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 4 01:52:16,789 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6595eea5 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} 01:52:16,789 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:52:16,789 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@39cf3758 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} 01:52:16,789 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 01:52:16,789 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:52:16,789 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 3 01:52:16,789 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@380aa2dc purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} 01:52:16,789 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 2 to 3 01:52:16,790 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 01:52:16,790 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], modifications=1, protocol=SIMPLE} with Envelope{sessionId=1, txSequence=1, message=TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0}} 01:52:16,790 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], modifications=1, protocol=SIMPLE}, enqueuedTicks=890077376455, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0}} 01:52:16,790 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 01:52:16,790 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], modifications=1, protocol=SIMPLE} completed with TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0} 01:52:16,790 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 01:52:16,790 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} directCommit completed 01:52:16,790 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 4 01:52:16,790 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR}: initiating purge 01:52:16,790 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 3 to 4 01:52:16,790 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 1 01:52:16,790 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 4 data Simple{size=241} 01:52:16,790 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1b14c609: creating a new snapshot member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0 01:52:16,790 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-740505897]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@1135ba67 01:52:16,790 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} has a successor, not scheduling timer 01:52:16,790 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-424264008], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:16,790 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SEALED}: initiating purge 01:52:16,790 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=DONE} allocated sequence 1 01:52:16,790 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-740505897]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@40da1472 01:52:16,790 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying foreign transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:52:16,790 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-39899300], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:16,790 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0 01:52:16,790 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2b85ecae transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-740505897]}, enqueuedTicks=891120691642} 01:52:16,790 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=OPEN} allocated sequence 0 01:52:16,790 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@6e009215 01:52:16,790 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2b85ecae transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=891121022120} 01:52:16,790 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 01:52:16,790 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:52:16,790 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 01:52:16,790 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 3 for term 27 01:52:16,791 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=4 01:52:16,791 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 73.30 μs 01:52:16,792 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: purging transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:52:16,792 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=5, term=27, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 01:52:16,792 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=27, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 01:52:16,793 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:52:16,793 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: allocating new open snapshot member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1 01:52:16,793 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges_slicer-id-713: slice: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, message: Envelope{sessionId=1, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0}} 01:52:16,793 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges_slicer-id-713: Message size: 378 bytes, total slices to send: 1 01:52:16,793 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges_slicer-id-713: Message does not need to be sliced - sending original message 01:52:16,793 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=5, term=27, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 01:52:16,794 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=1, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0}} 01:52:16,794 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=891121022120, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0}} 01:52:16,794 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0} 01:52:16,795 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} 01:52:16,795 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Issuing gracefulStop to shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:16,795 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges - waiting on 1 shards 01:52:16,795 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onShutDown 01:52:16,795 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Initiating leader transfer 01:52:16,795 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@58a83c6 01:52:16,795 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=13] 01:52:16,795 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@58a83c6 01:52:16,795 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Attempting to transfer leadership 01:52:16,795 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:16,795 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:52:16,795 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 3 01:52:16,795 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:16,795 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=4, prevLogTerm=27, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=27, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}]]] 01:52:16,797 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:52:16,797 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=13] 01:52:16,797 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=4, prevLogTerm=27, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=27, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}]]] 01:52:16,797 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 01:52:16,797 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 01:52:16,797 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75} 01:52:16,797 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=27, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 01:52:16,797 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 2 01:52:16,797 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=27, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=5, logLastTerm=27, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:52:16,797 PM [cluster-test-shard-dispatcher-16] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:16,797 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:52:16,797 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2b85ecae superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@30f9fdf, splicing queue 01:52:16,797 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-39899300], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 01:52:16,797 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2b85ecae queue spliced 1 messages 01:52:16,797 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-39899300], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-39899300], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:16,797 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: refreshing backend for shard 1 01:52:16,797 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-39899300], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:52:16,797 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:16,797 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:16,797 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#1653348113] 01:52:16,797 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=5, term=27, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 01:52:16,797 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:16,797 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:52:16,799 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 5, nextIndex: 6 01:52:16,799 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 01:52:16,799 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 5 01:52:16,799 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 4 to 5 01:52:16,799 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75} 01:52:16,799 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: replication of member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@3ae503a7 01:52:16,799 PM [cluster-test-shard-dispatcher-25] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: finished purging transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:52:16,799 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 01:52:16,799 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:52:16,799 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 4 for term 27 01:52:16,799 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 01:52:16,799 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, matchIndex: 5, lastIndex: 5, isVoting: true 01:52:16,799 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Follower's log matches - sending ElectionTimeout 01:52:16,799 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 01:52:16,799 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Leader transfer complete 01:52:16,799 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=5 01:52:16,800 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=2, message=TransactionPurgeResponse{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1}} not found in inflight queue, checking pending queue 01:52:16,800 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-740505897]} with Envelope{sessionId=1, txSequence=2, message=TransactionPurgeResponse{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1}} 01:52:16,800 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-740505897]}, enqueuedTicks=891120691642, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionPurgeResponse{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1}} 01:52:16,800 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=DONE}: purge completed 01:52:16,800 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6595eea5 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=DONE} 01:52:16,800 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR}: purge completed 01:52:16,800 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@39cf3758 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} 01:52:16,800 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 5 01:52:16,800 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5 01:52:16,800 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 5 01:52:16,800 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 4 to 5 01:52:16,800 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75} 01:52:16,800 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 01:52:16,800 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 01:52:16,800 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 4 for term 27 01:52:16,800 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received TimeoutNow - switching to Candidate 01:52:16,800 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:52:16,800 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=28, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:16,800 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 28 01:52:16,801 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=28, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=5, lastLogTerm=27] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:16,801 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 28 01:52:16,801 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:52:16,801 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:52:16,801 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:52:16,801 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 01:52:16,801 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 01:52:16,801 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=-1] 01:52:16,801 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized - notifying 1 OnShardInitialized callbacks 01:52:16,801 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 01:52:16,801 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized - notifying 1 OnShardInitialized callbacks 01:52:16,801 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=5 01:52:16,802 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:16,802 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Cluster state: [] 01:52:16,802 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Candidate in requestVote:RequestVote [term=28, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=5, lastLogTerm=27] with higher term appears reachable, updating term. 01:52:16,802 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Term 28 in "RequestVote [term=28, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=5, lastLogTerm=27]" message is greater than leader's term 27 - switching to Follower 01:52:16,802 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=28, votedFor=null 01:52:16,802 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Leadership transfer in progress - processing RequestVote 01:52:16,802 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): In requestVote: RequestVote [term=28, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=5, lastLogTerm=27] - currentTerm: 28, votedFor: null, lastIndex: 5, lastTerm: 27 01:52:16,802 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=28, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:16,802 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): requestVote returning: RequestVoteReply [term=28, voteGranted=true] 01:52:16,802 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader) :- Switching from behavior Leader to Follower, election term: 28 01:52:16,802 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-715: Closing 01:52:16,802 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 01:52:16,802 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 01:52:16,802 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 01:52:16,802 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Leader to Follower 01:52:16,802 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=-1] 01:52:16,802 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Leader to Follower 01:52:16,802 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=28, voteGranted=true], my term is 28 01:52:16,802 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=28, voteGranted=true], current voteCount: 1 01:52:16,802 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 01:52:16,802 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 4 01:52:16,802 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Leader, election term: 28 01:52:16,802 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 01:52:16,803 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges] 01:52:16,803 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Leader 01:52:16,803 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 01:52:16,803 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized - notifying 1 OnShardInitialized callbacks 01:52:16,803 PM [cluster-test-shard-dispatcher-16] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:16,803 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Leader 01:52:16,803 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is ready - notifying 1 OnShardInitialized callbacks 01:52:16,803 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1402098955, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@234717af] 01:52:16,803 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 01:52:16,803 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:52:16,803 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1402098955, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@234717af] 01:52:16,803 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@300abd48, attempting to connect 01:52:16,803 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 01:52:16,803 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:16,803 PM [cluster-test-shard-dispatcher-27] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Successfully transferred leadership to member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges in 8.248 ms 01:52:16,803 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: leader transfer succeeded - sending PoisonPill 01:52:16,803 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 01:52:16,803 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:16,803 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:52:16,803 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 01:52:16,803 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 01:52:16,804 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:16,804 PM [cluster-test-shard-dispatcher-13] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 01:52:16,804 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 01:52:16,804 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:52:16,804 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All shards shutdown - sending PoisonPill to self 01:52:16,804 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1402098955], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:52:16,804 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:52:16,804 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-39899300], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1402098955], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:16,804 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6595eea5 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@166d1b43 01:52:16,804 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=SUCCESSOR} previous state OPEN 01:52:16,804 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:52:16,804 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@166d1b43 01:52:16,804 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:52:16,804 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1 01:52:16,804 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=OPEN} 01:52:16,804 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] LocalProxyTransaction - Not replaying IncrementTransactionSequenceRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-740505897], snapshotOnly=true} 01:52:16,804 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Incrementing sequence org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$IncrementSequence@46f52eb8 to successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=OPEN} 01:52:16,804 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1 replayed 1 successful requests 01:52:16,804 PM [main] [WARN] ActorUtils - ShardManager for testSingleShardTransactionsWithLeaderChanges data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:52:16,804 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:52:16,804 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1b14c609 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1402098955], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:16,804 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 01:52:16,804 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=SUCCESSOR} 01:52:16,805 PM [main] [WARN] ActorUtils - ShardManager for testSingleShardTransactionsWithLeaderChanges data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor82.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 01:52:16,805 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:52:16,805 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:52:16,805 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:16,805 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:16,804 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 5, nextIndex: 6 01:52:16,805 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@19d44547 01:52:16,806 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@679df23d superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@15607a0d, splicing queue 01:52:16,806 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@679df23d queue spliced 0 messages 01:52:16,806 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 01:52:16,806 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-39899300], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1402098955], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 1.825 ms 01:52:16,810 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:16,810 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:16,810 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:16,810 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:16,812 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:52:16,827 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 01:52:16,827 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 01:52:16,827 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageSlicer - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-716: Closing 01:52:16,827 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 01:52:16,827 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 01:52:16,828 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:16,828 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:17,821 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [3789429996515803130] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:52:17,841 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:17,841 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:17,842 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-1191921172140774911] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:52:17,843 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:52:17.845] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:17,851 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:17,851 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/11/2024 13:52:17.854] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:17,868 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:52:17,873 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4308209869364509850] 01:52:17,873 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:52:17,874 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:52:17,874 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:52:17,875 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:17,875 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:17,884 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:52:17,884 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:52:17,884 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:52:17,885 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:52:17,889 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-1640986753279349170] 01:52:17,889 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:52:17,890 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:17,890 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:52:17,890 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:17,890 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:17,908 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:52:17,920 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-5595440934249659131] 01:52:17,920 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:52:17,922 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:17,922 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:52:17,922 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:17,922 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:17,931 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:52:17,931 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:17,932 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:17,933 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 01:52:17,934 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:52:17,939 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:17,940 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 01:52:17,942 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: no initial generation override, starting from 0 01:52:17,942 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:52:17,942 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7c447d1c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1ccb8808 01:52:17,942 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3b2085eb 01:52:17,942 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1710165137942, None)) 01:52:17,943 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 01:52:17,943 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:17,943 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:52:17,943 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:17,943 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165137941,0,0)) 01:52:17,943 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6cfdabaf 01:52:17,943 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1ccb8808 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@253b0f70 01:52:17,944 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} started 01:52:17,944 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:52:17,944 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:17,944 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:17,944 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:17,944 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:17,944 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-1-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940] 01:52:17,944 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithIsolatedLeader, persistent : true 01:52:17,944 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:17,945 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#-671474871 created and ready for shard:member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:17,945 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#-671474871] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:17,945 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/shard-member-1:snapshot-read#1130441628] 01:52:17,945 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/shard-member-1:snapshot-read#1130441628] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:17,945 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:17,946 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 01:52:17,946 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 01:52:17,947 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:52:17,947 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:52:17,948 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:52:17,948 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 01:52:17,948 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 01:52:17,948 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithIsolatedLeader is initialized 01:52:17,948 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 01:52:17,948 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 01:52:17,948 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from null to Follower 01:52:17,949 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#1081013514] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:17,950 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 01:52:17,951 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: no initial generation override, starting from 0 01:52:17,951 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:52:17,951 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4a8d9883 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4cc0f20 01:52:17,951 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 01:52:17,951 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:17,951 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:52:17,951 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:17,951 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7f76608f 01:52:17,951 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1710165137951, None)) 01:52:17,952 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165137950,0,0)) 01:52:17,952 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4cc0f20 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@d785731 01:52:17,952 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} started 01:52:17,953 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@689833c6 01:52:17,953 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:52:17,953 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:17,953 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:17,953 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:17,953 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-1192765554] 01:52:17,953 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithIsolatedLeader, persistent : true 01:52:17,954 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-1192765554] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:17,954 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#-1622145088] 01:52:17,954 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#1480354474 created and ready for shard:member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:17,954 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 01:52:17,954 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#1480354474] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:17,954 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#-1622145088] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:17,956 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:52:17,957 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:52:17,959 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:52:17,959 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:52:17,959 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:52:17,959 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 01:52:17,959 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 01:52:17,959 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithIsolatedLeader is initialized 01:52:17,959 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 01:52:17,959 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:17,959 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 01:52:17,959 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-2-shard-cars-testTransactionWithIsolatedLeader from null to Follower 01:52:18,3 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] 01:52:18,3 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: findLocalShard : cars 01:52:18,3 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940] 01:52:18,176 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:52:18,176 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Received ElectionTimeout - switching to Candidate 01:52:18,176 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 01:52:18,176 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:18,176 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 1 01:52:18,176 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:52:18,176 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 01:52:18,176 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:52:18,176 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 01:52:18,176 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Follower to Candidate 01:52:18,176 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Follower to Candidate 01:52:18,476 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 01:52:18,476 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:18,476 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 2 01:52:18,525 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:52:18,705 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 01:52:18,705 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:18,705 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 3 01:52:18,886 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:52:18,886 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 01:52:18,886 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:18,887 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:52:18,887 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:18,887 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:18,887 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:18,887 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:52:18,887 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:52:18,894 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:52:18,894 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 01:52:18,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:18,894 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:52:18,894 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:18,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:18,940 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:52:18,966 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 01:52:18,966 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:18,966 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 4 01:52:18,966 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:18,967 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:18,967 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Cluster state: [] 01:52:18,967 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:52:18,967 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:52:18,967 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=null 01:52:18,967 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:52:18,967 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:18,967 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:52:18,968 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:52:18,968 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:52:18,968 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 01:52:18,968 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:52:18,968 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:52:18,968 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:52:18,968 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: became leader with frontend state for [] 01:52:18,968 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Candidate to Leader 01:52:18,969 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 01:52:18,969 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:18,969 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:52:18,969 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Candidate to Leader 01:52:18,969 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 01:52:18,969 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader: Closing 01:52:18,969 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithIsolatedLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:18,969 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done false 01:52:18,969 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 01:52:18,970 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:18,970 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:52:18,970 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 01:52:18,970 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: -1, nextIndex: 0 01:52:18,970 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:52:19,10 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@753912b7: creating a new transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:52:19,10 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1} 01:52:19,10 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:19,10 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:19,10 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] 01:52:19,10 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:19,10 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} shard 1 01:52:19,10 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6aabb631] 01:52:19,10 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:52:19,10 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:52:19,10 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@753912b7: creating a new transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 01:52:19,10 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6aabb631] 01:52:19,10 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 01:52:19,10 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:52:19,10 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@753912b7: creating a new transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:52:19,11 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:52:19,11 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=OPEN} allocated sequence 0 01:52:19,11 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@7356071c, attempting to connect 01:52:19,12 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} is not yet known 01:52:19,12 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, nanosAgo=14941, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:52:19,12 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:52:19,12 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:52:19,12 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:19,12 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1cdc84a7 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1b8b8943 01:52:19,12 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 01:52:19,12 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} previous state OPEN 01:52:19,12 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} previous state OPEN 01:52:19,12 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1b8b8943 01:52:19,12 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:52:19,12 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=OPEN} 01:52:19,12 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1b8b8943 01:52:19,12 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 01:52:19,12 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=OPEN} 01:52:19,12 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1b8b8943 01:52:19,12 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:52:19,12 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=OPEN} 01:52:19,12 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@753912b7 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:19,13 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} 01:52:19,13 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} 01:52:19,13 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} 01:52:19,13 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@7ba4760c 01:52:19,13 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7736ef03 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@a233790, splicing queue 01:52:19,13 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7736ef03 queue spliced 0 messages 01:52:19,13 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 595.4 μs 01:52:19,21 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:52:19,21 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:19,21 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 01:52:19,22 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onShutDown 01:52:19,22 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:19,22 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 01:52:19,22 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 01:52:19,22 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 01:52:19,22 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 01:52:19,22 PM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:52:19,23 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request ModifyTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], modifications=1} 01:52:19,23 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@753912b7 readied transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 01:52:19,24 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithIsolatedLeader 01:52:19,24 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] 01:52:19,24 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:19,24 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: shutting down 01:52:19,24 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:52:19,24 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: findLocalShard : cars 01:52:19,24 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940] 01:52:19,24 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 01:52:19,24 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#1081013514] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#1081013514] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:19,25 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#-504690733] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-504690733] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:19,25 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#1081013514] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#1081013514] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:19,25 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940] 01:52:19,75 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940] 01:52:19,86 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:19,126 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940] 01:52:19,176 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940] 01:52:19,205 PM [cluster-test-shard-dispatcher-23] [WARN] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: At least 1 followers need to be active, Switching member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 01:52:19,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 01:52:19,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:52:19,206 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 4 01:52:19,206 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader)_slicer-id-721: Closing 01:52:19,206 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:52:19,206 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Leader to IsolatedLeader 01:52:19,206 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 01:52:19,207 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:19,207 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:19,227 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940] 01:52:19,227 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request ModifyTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], modifications=1} 01:52:19,227 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@753912b7 readied transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 01:52:19,228 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=FLUSHED} allocated sequence 0 01:52:19,228 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@6a665bb8 01:52:19,228 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:52:19,228 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1aadfa3c transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], coordinated=false}, enqueuedTicks=893558633630} 01:52:19,228 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: not currently active leader, rejecting request Envelope{sessionId=0, txSequence=0, message=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], coordinated=false}}. isLeader: true, isLeaderActive: false,isLeadershipTransferInProgress: false, paused: false 01:52:19,228 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: request Envelope{sessionId=0, txSequence=0, message=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], coordinated=false}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:52:19,229 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} indicated no leadership, reconnecting it org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:52:19,229 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1aadfa3c superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@27a67fff, splicing queue 01:52:19,229 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:52:19,229 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1aadfa3c queue spliced 1 messages 01:52:19,229 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:19,229 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: refreshing backend for shard 1 01:52:19,229 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:52:19,229 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:19,229 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:19,229 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] 01:52:19,229 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:19,229 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 01:52:19,645 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 01:52:19,645 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 01:52:19,645 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:52:19,646 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:19,646 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:19,646 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:19,905 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 01:52:20,666 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:20,666 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:20,666 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:20,666 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] 01:52:20,666 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:20,666 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 01:52:21,85 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 01:52:21,85 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 01:52:21,85 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:52:21,86 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:21,86 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:21,86 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:22,106 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:22,106 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:22,106 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:22,106 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] 01:52:22,106 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:22,106 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 01:52:22,525 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 01:52:22,525 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 01:52:22,526 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:52:22,526 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:22,526 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:22,526 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:23,545 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:23,545 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:23,545 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:23,545 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] 01:52:23,546 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:23,546 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 01:52:23,965 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 01:52:23,965 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 01:52:23,965 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:52:23,966 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:23,966 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:23,966 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:24,986 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:24,986 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:24,986 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:24,986 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] 01:52:24,986 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:24,986 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 01:52:25,406 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 01:52:25,406 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 01:52:25,406 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:52:25,406 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:25,407 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:25,407 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:26,425 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:26,425 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:26,425 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:26,425 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] 01:52:26,425 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:26,425 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 01:52:26,845 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 01:52:26,845 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 01:52:26,845 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:52:26,846 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:26,846 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:26,846 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:27,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:27,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:27,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:27,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] 01:52:27,866 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:27,866 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 01:52:28,285 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 01:52:28,285 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 01:52:28,285 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:52:28,286 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:28,286 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:28,286 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:29,305 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:29,305 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:29,305 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:29,305 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] 01:52:29,306 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:29,306 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 01:52:29,725 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 01:52:29,725 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 01:52:29,726 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:52:29,726 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:29,726 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:29,726 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:30,745 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:30,745 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:30,745 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:30,745 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] 01:52:30,745 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:30,745 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 01:52:31,165 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 01:52:31,165 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 01:52:31,165 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 01:52:31,165 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:31,166 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:31,166 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 01:52:31,228 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithIsolatedLeader 01:52:31,229 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request ModifyTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], modifications=1} 01:52:31,229 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@753912b7 readied transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 01:52:31,229 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:31,229 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:31,229 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 01:52:31,231 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:31,233 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 01:52:31,235 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 01:52:31,236 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: recovered identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:52:31,236 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} 01:52:31,236 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6d57a87e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2f91c0a2 01:52:31,236 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4ad598a3 01:52:31,237 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 01:52:31,237 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1710165151237, None)) 01:52:31,237 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:31,237 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165151236,0,0)) 01:52:31,237 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:52:31,237 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 01:52:31,237 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:52:31,237 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2f91c0a2 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4da91ed2 01:52:31,237 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:31,237 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:52:31,237 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:31,237 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} started 01:52:31,237 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@67be1676 01:52:31,237 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:52:31,237 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=FLUSHED} allocated sequence 0 01:52:31,237 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:31,238 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@508cabf 01:52:31,238 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:31,238 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:31,238 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:31,238 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#517428290] 01:52:31,238 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:31,238 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:31,238 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-2-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#517428290] 01:52:31,238 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 01:52:31,238 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:31,238 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-3-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#517428290] 01:52:31,238 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithIsolatedLeader, persistent : true 01:52:31,239 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 01:52:31,239 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#708518817] 01:52:31,239 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#-688353845 created and ready for shard:member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:31,239 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:52:31,239 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=null 01:52:31,239 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 01:52:31,239 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:31,239 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:52:31,239 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:52:31,240 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 01:52:31,240 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 01:52:31,240 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 01:52:31,240 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithIsolatedLeader is initialized 01:52:31,240 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 01:52:31,240 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:31,240 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 01:52:31,240 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-2-shard-cars-testTransactionWithIsolatedLeader from null to Follower 01:52:31,327 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader: Closing 01:52:31,327 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithIsolatedLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:31,327 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 01:52:31,327 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:31,327 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:52:31,327 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 01:52:31,327 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done false 01:52:31,328 PM [cluster-test-shard-dispatcher-23] [WARN] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 12358, lastApplied : -1, commitIndex : -1 01:52:31,328 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - IsolatedLeader member-1-shard-cars-testTransactionWithIsolatedLeader switching from IsolatedLeader to Leader 01:52:31,328 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 01:52:31,328 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:52:31,328 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 4 01:52:31,328 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader)_slicer-id-722: Closing 01:52:31,328 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 01:52:31,328 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from IsolatedLeader to Leader 01:52:31,328 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from IsolatedLeader to Leader 01:52:31,328 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 01:52:31,329 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done true 01:52:32,185 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:32,186 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:32,186 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:32,186 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] 01:52:32,186 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:32,186 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6aabb631] 01:52:32,186 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6aabb631] 01:52:32,186 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@5a15b0de, attempting to connect 01:52:32,187 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 01:52:32,187 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:52:32,187 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:32,187 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1b8b8943 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@77760030 01:52:32,187 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:52:32,187 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} previous state FLUSHED 01:52:32,187 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} previous state SEALED 01:52:32,187 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@77760030 01:52:32,187 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:52:32,187 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=OPEN} 01:52:32,187 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], coordinated=false} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=OPEN} 01:52:32,187 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SEALED} allocated sequence 0 01:52:32,187 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@508cabf 01:52:32,187 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:52:32,187 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@70ed1e48 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], coordinated=false}, enqueuedTicks=905568585276} 01:52:32,188 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@77760030 01:52:32,188 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 01:52:32,188 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=OPEN} 01:52:32,188 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], coordinated=false} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=OPEN} 01:52:32,188 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SEALED} allocated sequence 0 01:52:32,188 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@6a665bb8 01:52:32,188 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@70ed1e48 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], coordinated=false}, enqueuedTicks=893558633630} 01:52:32,188 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@77760030 01:52:32,188 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:52:32,188 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=OPEN} 01:52:32,188 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=OPEN} 01:52:32,188 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], modifications=1} 01:52:32,188 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@753912b7 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:32,188 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} 01:52:32,188 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} 01:52:32,188 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} 01:52:32,188 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@438aca97 01:52:32,188 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@399d5d20 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@739bd45, splicing queue 01:52:32,188 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@399d5d20 queue spliced 0 messages 01:52:32,188 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 1.363 ms 01:52:32,188 PM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: allocating new ready transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:52:32,188 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 initiating direct canCommit 01:52:32,188 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:52:32,188 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 validated 01:52:32,188 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:52:32,188 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 initiating direct preCommit 01:52:32,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Preparing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:52:32,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 candidate ready 01:52:32,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1: canCommit - candidate rootPath: / 01:52:32,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1: canCommit - messages: [] 01:52:32,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 01:52:32,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 prepared 01:52:32,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:52:32,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 initiating direct commit 01:52:32,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Starting commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:52:32,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 171 01:52:32,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 01:52:32,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 01:52:32,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:52:32,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:52:32,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:32,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}]]] 01:52:32,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 submitted to persistence 01:52:32,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: allocating new ready transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 01:52:32,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 initiating direct canCommit 01:52:32,189 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 01:52:32,190 PM [cluster-test-shard-dispatcher-23] [WARN] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Store Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1: Conflicting modification for path /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars. 01:52:32,190 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to FAILED 01:52:32,190 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], coordinated=false} with Envelope{sessionId=1, txSequence=1, message=TransactionFailure{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: CanCommit failed}} 01:52:32,190 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], coordinated=false}, enqueuedTicks=893558633630, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionFailure{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: CanCommit failed}} 01:52:32,190 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} directCommit completed 01:52:32,190 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR}: initiating purge 01:52:32,190 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 1 01:52:32,190 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-276040164]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@6f0c58f6 01:52:32,190 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:32,190 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:32,190 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: CanCommit failed at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction$8.onFailure(FrontendReadWriteTransaction.java:441) at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.failedCanCommit(SimpleShardDataTreeCohort.java:175) at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$4(ShardDataTree.java:862) at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:884) at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:831) at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:986) at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:99) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.directCommit(FrontendReadWriteTransaction.java:433) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleCommitLocalTransaction(FrontendReadWriteTransaction.java:524) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:193) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: OptimisticLockFailedException{message=Optimistic lock failed for path /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, errorList=[RpcError [message=Optimistic lock failed for path /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, severity=ERROR, errorType=APPLICATION, tag=resource-denied, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.tree.api.ConflictingModificationAppliedException: Node was created by other transaction.]]} at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$4(ShardDataTree.java:846) ... 41 more Caused by: org.opendaylight.yangtools.yang.data.tree.api.ConflictingModificationAppliedException: Node was created by other transaction. at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.checkConflicting(SchemaAwareApplyOperation.java:74) at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.checkWriteApplicable(SchemaAwareApplyOperation.java:181) at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:111) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:409) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:368) at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:110) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractDataTreeTip.validate(AbstractDataTreeTip.java:31) at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$4(ShardDataTree.java:838) ... 41 more 01:52:32,191 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 01:52:32,190 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-276040164]} 01:52:32,192 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SEALED}: initiating purge 01:52:32,192 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=DONE} allocated sequence 1 01:52:32,192 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-276040164]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@540e9f29 01:52:32,192 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@70ed1e48 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-276040164]}, enqueuedTicks=906520975331} 01:52:32,193 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 01:52:32,193 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 01:52:32,193 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 01:52:32,193 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:52:32,193 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}]]] 01:52:32,193 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 1 01:52:32,193 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:52:32,193 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log Simple{size=171} 01:52:32,193 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 01:52:32,193 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 1 01:52:32,193 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:52:32,193 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 01:52:32,193 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 01:52:32,198 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 0, nextIndex: 1 01:52:32,198 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:52:32,198 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 0 01:52:32,198 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from -1 to 0 01:52:32,198 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 0 data Simple{size=171} 01:52:32,198 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Resuming commit of transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:52:32,198 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionWithIsolatedLeader: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@6d07621c}, purged=MutableUnsignedLongSet{size=0}} 01:52:32,198 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 01:52:32,198 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Committed transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:52:32,198 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1: commit - successfulFromPrevious: [] 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionWithIsolatedLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#1992280113] with name member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher 01:52:32,199 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], coordinated=false} with Envelope{sessionId=1, txSequence=0, message=TransactionCommitSuccess{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0}} 01:52:32,199 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], coordinated=false}, enqueuedTicks=905568585276, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCommitSuccess{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0}} 01:52:32,199 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} directCommit completed 01:52:32,199 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 01:52:32,199 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:32,199 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-276040164]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@33570e17 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}]]] 01:52:32,199 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=FLUSHED} allocated sequence 0 01:52:32,199 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=0 01:52:32,199 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:32,199 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x0000000801307d30@7f4c0d4b 01:52:32,199 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#1992280113] 01:52:32,199 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@70ed1e48 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], coordinated=false}, enqueuedTicks=906529966054} 01:52:32,199 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-276040164]} 01:52:32,199 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SEALED}: initiating purge 01:52:32,199 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=DONE} allocated sequence 1 01:52:32,199 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 01:52:32,199 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-276040164]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@4a4f9889 01:52:32,199 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@70ed1e48 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-276040164]}, enqueuedTicks=906529793749} 01:52:32,199 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 48.04 μs 01:52:32,199 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#1992280113] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: allocating new ready transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 initiating direct canCommit 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 validated 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 initiating direct preCommit 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Preparing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 candidate ready 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1: canCommit - candidate rootPath: / 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1: canCommit - messages: [] 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1: preCommit - successfulFromPrevious: [] 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 prepared 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 initiating direct commit 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Starting commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 69 01:52:32,199 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}] 01:52:32,200 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}] 01:52:32,200 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:52:32,200 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 01:52:32,200 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:32,200 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 submitted to persistence 01:52:32,200 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:52:32,200 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 01:52:32,200 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 01:52:32,200 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:52:32,201 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}] 01:52:32,201 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 01:52:32,201 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}]]] 01:52:32,201 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 1 01:52:32,201 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:52:32,201 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64} 01:52:32,201 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 01:52:32,201 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 2 01:52:32,201 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 0 01:52:32,201 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:52:32,201 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 01:52:32,201 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 0 01:52:32,201 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from -1 to 0 01:52:32,201 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 0 data Simple{size=171} 01:52:32,201 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying foreign transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:52:32,202 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionWithIsolatedLeader: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@201b044b}, purged=MutableUnsignedLongSet{size=0}} 01:52:32,202 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 01:52:32,202 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Committed transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:52:32,202 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionWithIsolatedLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-955851110] with name member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher 01:52:32,203 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-955851110] 01:52:32,203 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 01:52:32,203 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 73.75 μs 01:52:32,203 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 01:52:32,203 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=0 01:52:32,203 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 1, nextIndex: 2 01:52:32,203 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:52:32,203 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 1 01:52:32,203 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 0 to 1 01:52:32,203 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64} 01:52:32,203 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: replication of member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@49254b0d 01:52:32,203 PM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: finished purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 01:52:32,203 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 01:52:32,203 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 01:52:32,203 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-276040164]} with Envelope{sessionId=1, txSequence=2, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1}} 01:52:32,203 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 0 for term 4 01:52:32,203 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-276040164]}, enqueuedTicks=906520975331, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1}} 01:52:32,203 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=DONE}: purge completed 01:52:32,203 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 01:52:32,203 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@77760030 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=DONE} 01:52:32,203 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:32,203 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR}: purge completed 01:52:32,203 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1b8b8943 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} 01:52:32,203 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}], SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}]]] 01:52:32,203 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=1 01:52:32,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}], SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}]]] 01:52:32,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 2 01:52:32,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 01:52:32,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log Simple{size=69} 01:52:32,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}] 01:52:32,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63} 01:52:32,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 01:52:32,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 4 01:52:32,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 1 01:52:32,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:52:32,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 3 01:52:32,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 1 01:52:32,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 0 to 1 01:52:32,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64} 01:52:32,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 01:52:32,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 01:52:32,204 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 0 for term 4 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 3, nextIndex: 4 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 2 01:52:32,205 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}] 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 1 to 2 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 2 data Simple{size=69} 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Resuming commit of transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:52:32,205 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 01:52:32,205 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=1 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Committed transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1: commit - successfulFromPrevious: [] 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 changing state from COMMIT_PENDING to COMMITTED 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 3 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 2 to 3 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63} 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: replication of member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@305b4aab 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: finished purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:52:32,205 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], coordinated=false} with Envelope{sessionId=1, txSequence=3, message=TransactionCommitSuccess{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0}} 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 01:52:32,205 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-276040164], coordinated=false}, enqueuedTicks=906529966054, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionCommitSuccess{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0}} 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 2 for term 4 01:52:32,205 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=FLUSHED} directCommit completed 01:52:32,205 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=FLUSHED}: initiating purge 01:52:32,205 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=DONE} allocated sequence 1 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 01:52:32,205 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-276040164]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@4789b2de 01:52:32,205 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@70ed1e48 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-276040164]}, enqueuedTicks=906535965802} 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=3 01:52:32,205 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-276040164]} with Envelope{sessionId=1, txSequence=4, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1}} 01:52:32,205 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-276040164]}, enqueuedTicks=906529793749, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1}} 01:52:32,205 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=DONE}: purge completed 01:52:32,205 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 01:52:32,205 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@77760030 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=DONE} 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:52:32,205 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:52:32,205 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1b8b8943 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:52:32,205 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 26.53 μs 01:52:32,205 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 01:52:32,206 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:52:32,206 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:32,206 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 01:52:32,206 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onShutDown 01:52:32,206 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Initiating leader transfer 01:52:32,206 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@213c5cb8 01:52:32,206 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderId=null, leaderPayloadVersion=13] 01:52:32,206 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@213c5cb8 01:52:32,207 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Attempting to transfer leadership 01:52:32,207 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 2 01:52:32,207 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:32,207 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 01:52:32,207 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:32,207 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:52:32,207 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@70ed1e48 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@28e61165, splicing queue 01:52:32,207 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 01:52:32,207 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 3 01:52:32,207 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@70ed1e48 queue spliced 1 messages 01:52:32,207 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 01:52:32,207 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 2 01:52:32,207 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 1 to 2 01:52:32,207 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 2 data Simple{size=69} 01:52:32,207 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 01:52:32,207 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying foreign transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:52:32,207 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: refreshing backend for shard 1 01:52:32,207 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 01:52:32,207 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#928977940], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 01:52:32,207 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Committed transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:52:32,207 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 3 01:52:32,207 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 2 to 3 01:52:32,207 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63} 01:52:32,207 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:32,207 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:32,207 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 01:52:32,207 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] 01:52:32,207 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 01:52:32,207 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 2 for term 4 01:52:32,207 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=3 01:52:32,207 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 01:52:32,207 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 13.88 μs 01:52:32,207 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:32,207 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 20000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 01:52:32,208 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:32,208 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionWithIsolatedLeader, matchIndex: 3, lastIndex: 4, isVoting: true 01:52:32,208 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader] 01:52:32,208 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 01:52:32,208 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 1 01:52:32,208 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 01:52:32,208 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64} 01:52:32,208 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 01:52:32,208 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 2 01:52:32,208 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:52:32,209 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=null, leaderPayloadVersion=13] 01:52:32,209 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 01:52:32,209 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:32,209 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:52:32,209 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 4, nextIndex: 5 01:52:32,209 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 01:52:32,209 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 4 01:52:32,209 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 3 to 4 01:52:32,209 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64} 01:52:32,209 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: replication of member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@2c801252 01:52:32,209 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: finished purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:52:32,209 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 01:52:32,209 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:52:32,209 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 3 for term 4 01:52:32,209 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:52:32,209 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionWithIsolatedLeader, matchIndex: 4, lastIndex: 4, isVoting: true 01:52:32,209 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Follower's log matches - sending ElectionTimeout 01:52:32,210 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 01:52:32,210 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Leader transfer complete 01:52:32,210 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=4 01:52:32,210 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 4 01:52:32,210 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4 01:52:32,210 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 4 01:52:32,210 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 3 to 4 01:52:32,210 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64} 01:52:32,210 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1}} not found in inflight queue, checking pending queue 01:52:32,210 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 01:52:32,210 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 01:52:32,210 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-276040164]} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1}} 01:52:32,210 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 3 for term 4 01:52:32,210 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-276040164]}, enqueuedTicks=906535965802, sessionId=1, txSequence=5} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1}} 01:52:32,210 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=DONE}: purge completed 01:52:32,210 PM [cluster-test-shard-dispatcher-32] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:52:32,210 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@77760030 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=DONE} 01:52:32,210 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=4 01:52:34,225 PM [cluster-test-shard-dispatcher-22] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testTransactionWithIsolatedLeader: Successfully transferred leadership to member-1-shard-cars-testTransactionWithIsolatedLeader in 2.019 s 01:52:34,225 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: leader transfer succeeded - sending PoisonPill 01:52:34,226 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithIsolatedLeader 01:52:34,226 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader)_slicer-id-725: Closing 01:52:34,226 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 01:52:34,226 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 01:52:34,226 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 01:52:34,226 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:52:34,226 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 01:52:34,226 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: shutting down 01:52:34,227 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:52:34,227 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:52:34,227 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:34,227 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#-276040164] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-276040164] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:34,227 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1173/0x00000008012cced0] to Actor[akka://cluster-test/user/$a#-276040164] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-276040164] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:34,227 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:34,227 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:34,227 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#2140609907] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:34,227 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:34,232 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:34,232 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:34,232 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:34,233 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:34,235 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 01:52:34,247 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 01:52:34,249 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithIsolatedLeader 01:52:34,249 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 01:52:34,249 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 01:52:34,249 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 01:52:34,249 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:34,249 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:35,240 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4308209869364509850] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:52:35,266 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:35,266 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:35,267 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:52:35.268] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:35,270 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-1640986753279349170] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:52:35,276 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:35,276 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:35,277 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:52:35.278] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:35,288 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 01:52:35,294 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-6705261226330559508] 01:52:35,294 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 01:52:35,296 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 01:52:35,296 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 01:52:35,296 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:35,296 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:35,312 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 01:52:35,312 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 01:52:35,312 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 01:52:35,313 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 01:52:35,318 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-3281023276872059784] 01:52:35,318 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 01:52:35,319 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:35,319 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 01:52:35,329 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:35,329 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:35,334 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 01:52:35,342 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [3562378891230898094] 01:52:35,342 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 01:52:35,351 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:35,351 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 01:52:35,351 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 01:52:35,351 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 01:52:35,355 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:35,356 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:35,357 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithMultipleShards 01:52:35,363 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-1445786069] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:35,364 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithMultipleShards 01:52:35,364 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: no initial generation override, starting from 0 01:52:35,364 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:52:35,364 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@47c5dd98 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@72554815 01:52:35,364 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@14a644e 01:52:35,364 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testWriteTransactionWithMultipleShards, 0, 1710165155364, None)) 01:52:35,364 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithMultipleShards 01:52:35,364 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-cars-testWriteTransactionWithMultipleShards 01:52:35,364 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165155363,0,0)) 01:52:35,365 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-people-testWriteTransactionWithMultipleShards 01:52:35,365 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@72554815 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@70b23f0 01:52:35,365 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:52:35,365 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:52:35,365 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:52:35,365 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} started 01:52:35,365 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6603d20a 01:52:35,365 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:52:35,365 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testWriteTransactionWithMultipleShards 01:52:35,365 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:35,365 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:52:35,365 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:52:35,365 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards on shard member-1-shard-cars-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-804479436] 01:52:35,365 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testWriteTransactionWithMultipleShards 01:52:35,365 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Shard created : member-1-shard-cars-testWriteTransactionWithMultipleShards, persistent : true 01:52:35,366 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-804479436] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:35,366 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Starting recovery for member-1-shard-cars-testWriteTransactionWithMultipleShards with journal batch size 1 01:52:35,366 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Shard created : member-1-shard-people-testWriteTransactionWithMultipleShards, persistent : true 01:52:35,366 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 01:52:35,366 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Starting recovery for member-1-shard-people-testWriteTransactionWithMultipleShards with journal batch size 1 01:52:35,366 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-250850145] 01:52:35,366 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-notifier#-1072460704 created and ready for shard:member-1-shard-people-testWriteTransactionWithMultipleShards 01:52:35,366 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1015240369] 01:52:35,367 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-notifier#-1465606429] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:35,367 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-notifier#-1465606429 created and ready for shard:member-1-shard-cars-testWriteTransactionWithMultipleShards 01:52:35,368 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithMultipleShards 01:52:35,368 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1726422422] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:35,368 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-250850145] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:35,368 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-notifier#-1072460704] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:35,368 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1015240369] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:35,371 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:52:35,371 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:52:35,371 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:52:35,371 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:52:35,371 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:52:35,371 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:52:35,371 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards on shard member-1-shard-people-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1726422422] 01:52:35,371 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:52:35,371 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 01:52:35,371 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [people] 01:52:35,371 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:52:35,371 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-testWriteTransactionWithMultipleShards is initialized 01:52:35,371 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [cars] 01:52:35,371 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-testWriteTransactionWithMultipleShards is initialized 01:52:35,371 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 01:52:35,371 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from null to Follower 01:52:35,371 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 01:52:35,371 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from null to Follower 01:52:35,371 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from null to Follower 01:52:35,371 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 01:52:35,371 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from null to Follower 01:52:35,378 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-2115265394] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:35,379 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithMultipleShards 01:52:35,380 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 01:52:35,380 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithMultipleShards 01:52:35,380 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: no initial generation override, starting from 0 01:52:35,380 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:52:35,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-cars-testWriteTransactionWithMultipleShards 01:52:35,381 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@27dadc6e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@74750a7f 01:52:35,381 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@30748065 01:52:35,381 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-people-testWriteTransactionWithMultipleShards 01:52:35,381 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithMultipleShards, 0, 1710165155381, None)) 01:52:35,381 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1710165155380,0,0)) 01:52:35,382 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@290332d9 01:52:35,382 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@74750a7f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5da101c0 01:52:35,382 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} started 01:52:35,383 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:52:35,384 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 01:52:35,384 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithMultipleShards 01:52:35,384 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testWriteTransactionWithMultipleShards 01:52:35,384 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 01:52:35,384 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:52:35,384 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:52:35,384 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:52:35,384 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:52:35,384 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-people-testWriteTransactionWithMultipleShards, persistent : true 01:52:35,384 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithMultipleShards, persistent : true 01:52:35,384 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards#-539699067] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:35,384 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards#312690796] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:35,385 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-notifier#471015384 created and ready for shard:member-2-shard-people-testWriteTransactionWithMultipleShards 01:52:35,385 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-notifier#471015384] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:35,385 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-718427788] 01:52:35,385 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-718427788] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:35,385 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#442276356] 01:52:35,385 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#442276356] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:35,386 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 01:52:35,387 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 01:52:35,391 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-people-testWriteTransactionWithMultipleShards with journal batch size 1 01:52:35,391 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithMultipleShards with journal batch size 1 01:52:35,392 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-notifier#-675303018 created and ready for shard:member-2-shard-cars-testWriteTransactionWithMultipleShards 01:52:35,392 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-notifier#-675303018] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:35,396 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:52:35,396 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:52:35,396 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [people] 01:52:35,396 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-testWriteTransactionWithMultipleShards is initialized 01:52:35,396 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 01:52:35,396 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 01:52:35,396 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:52:35,396 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 01:52:35,396 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-people-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:52:35,396 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [cars] 01:52:35,396 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithMultipleShards is initialized 01:52:35,396 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithMultipleShards , received role change from null to Follower 01:52:35,396 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 01:52:35,396 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-2-shard-people-testWriteTransactionWithMultipleShards from null to Follower 01:52:35,397 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:52:35,397 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 01:52:35,397 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:52:35,397 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithMultipleShards , received role change from null to Follower 01:52:35,397 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 01:52:35,397 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-2-shard-cars-testWriteTransactionWithMultipleShards from null to Follower 01:52:35,432 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-1445786069] 01:52:35,432 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: findLocalShard : cars 01:52:35,432 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-804479436] 01:52:35,604 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:52:35,604 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:52:35,604 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testWriteTransactionWithMultipleShards] 01:52:35,604 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 01:52:35,604 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 01:52:35,604 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 01:52:35,604 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 01:52:35,604 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testWriteTransactionWithMultipleShards] 01:52:35,604 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 01:52:35,604 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:52:35,604 PM [cluster-test-shard-dispatcher-15] [DEBUG] MessageAssembler - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Closing 01:52:35,604 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 01:52:35,604 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:52:35,604 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 01:52:35,604 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 01:52:35,604 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Closing 01:52:35,604 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 01:52:35,604 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 01:52:35,604 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from Follower to Candidate 01:52:35,604 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from Follower to Candidate 01:52:35,604 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from Follower to Candidate 01:52:35,604 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from Follower to Candidate 01:52:35,855 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:52:35,855 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 01:52:35,855 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 01:52:35,884 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:52:35,884 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 01:52:35,884 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 01:52:35,942 PM [cluster-test-shard-dispatcher-32] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:52:35,981 PM [cluster-test-shard-dispatcher-32] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 01:52:36,164 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:52:36,164 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 01:52:36,164 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 01:52:36,174 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:52:36,174 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 01:52:36,174 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 01:52:36,315 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 01:52:36,315 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 01:52:36,316 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:52:36,316 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards 01:52:36,316 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:52:36,316 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards 01:52:36,316 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:52:36,316 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:52:36,316 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:52:36,316 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:52:36,316 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:52:36,316 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:52:36,316 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-cars-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:52:36,316 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Peer address for peer member-2-shard-people-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:52:36,320 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 01:52:36,321 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards 01:52:36,321 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:52:36,321 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards 01:52:36,321 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:52:36,321 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 01:52:36,321 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:52:36,321 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 01:52:36,321 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:52:36,321 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 01:52:36,359 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 01:52:36,464 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:52:36,464 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 01:52:36,464 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 01:52:36,464 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testWriteTransactionWithMultipleShards 01:52:36,466 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:36,466 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Cluster state: [] 01:52:36,466 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:52:36,466 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:52:36,466 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 01:52:36,466 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:52:36,466 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 01:52:36,466 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:52:36,467 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:52:36,467 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:52:36,467 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testWriteTransactionWithMultipleShards] 01:52:36,467 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:52:36,467 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:52:36,467 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:52:36,467 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: became leader with frontend state for [] 01:52:36,467 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:52:36,467 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from Candidate to Leader 01:52:36,467 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:52:36,467 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:52:36,467 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from Candidate to Leader 01:52:36,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithMultipleShards: Closing 01:52:36,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 01:52:36,468 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testWriteTransactionWithMultipleShards status sync done false 01:52:36,468 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:52:36,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:52:36,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:52:36,468 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:52:36,469 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:52:36,474 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 01:52:36,474 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 01:52:36,474 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 01:52:36,474 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testWriteTransactionWithMultipleShards 01:52:36,475 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 01:52:36,475 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Cluster state: [] 01:52:36,475 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 01:52:36,475 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 01:52:36,475 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 01:52:36,475 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 01:52:36,475 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 01:52:36,475 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 01:52:36,476 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 01:52:36,476 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 01:52:36,476 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testWriteTransactionWithMultipleShards] 01:52:36,476 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:52:36,476 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 01:52:36,476 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 01:52:36,476 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: became leader with frontend state for [] 01:52:36,476 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from Candidate to Leader 01:52:36,476 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:52:36,476 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:36,476 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:52:36,476 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from Candidate to Leader 01:52:36,476 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All Shards are ready - data store testWriteTransactionWithMultipleShards is ready 01:52:36,476 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithMultipleShards: Closing 01:52:36,476 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 01:52:36,476 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithMultipleShards status sync done false 01:52:36,476 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 01:52:36,476 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:36,476 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:52:36,476 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All Shards are ready - data store testWriteTransactionWithMultipleShards is ready 01:52:36,477 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 01:52:36,477 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 01:52:36,539 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-1445786069] 01:52:36,539 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: findLocalShard : people 01:52:36,539 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1726422422] 01:52:36,540 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@770d3790: creating a new transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:52:36,540 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} 01:52:36,540 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:36,540 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:36,540 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-2115265394] 01:52:36,540 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} shard 1 01:52:36,540 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:52:36,540 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:36,540 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:52:36,540 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards, primaryVersion=13] 01:52:36,540 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} 01:52:36,540 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:52:36,540 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:52:36,540 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-2115265394] 01:52:36,540 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} shard 2 01:52:36,540 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards, primaryVersion=13] 01:52:36,540 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:52:36,540 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:52:36,540 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@770d3790 readied transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 01:52:36,541 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@1358ef95 01:52:36,541 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} scheduling timeout in 30 seconds 01:52:36,541 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:52:36,541 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1228/0x0000000801307220@2b18be33 01:52:36,541 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@4ffdd169, attempting to connect 01:52:36,541 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} scheduling timeout in 30 seconds 01:52:36,541 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards, primaryVersion=13] 01:52:36,541 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards, primaryVersion=13] 01:52:36,541 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@8ed0fba, attempting to connect 01:52:36,542 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} is not yet known 01:52:36,542 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, nanosAgo=13281, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:52:36,542 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} is not yet known 01:52:36,542 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, nanosAgo=13691, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:52:36,543 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:52:36,543 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 01:52:36,543 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-804479436], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:52:36,543 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-804479436], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:36,543 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@639a5e7e instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@352da49a 01:52:36,543 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 01:52:36,543 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@352da49a 01:52:36,543 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:52:36,543 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} 01:52:36,543 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} 01:52:36,543 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2239/0x00000008016abcb8@644c5b13 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-804479436], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6187feb0 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=910871579033} 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-729: slice: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], modifications=1, protocol=THREE_PHASE}} 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-729: Message size: 391 bytes, total slices to send: 1 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-729: Message does not need to be sliced - sending original message 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@770d3790 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-804479436], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@27508be3 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6f2fbb32 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@2cd17dcd, splicing queue 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6f2fbb32 queue spliced 0 messages 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-804479436], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 654.3 μs 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1726422422], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1726422422], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6ff31428 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1b256c2 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} previous state FLUSHED 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1b256c2 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2239/0x00000008016abcb8@34cea35f 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1726422422], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@60730a61 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=910871667725} 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-729: slice: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], modifications=1, protocol=THREE_PHASE}} 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-729: Message size: 396 bytes, total slices to send: 1 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-729: Message does not need to be sliced - sending original message 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@770d3790 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1726422422], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@51c113c2 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4bdd5c3 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@12026350, splicing queue 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4bdd5c3 queue spliced 0 messages 01:52:36,544 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1726422422], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 400.9 μs 01:52:36,545 PM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithMultipleShards: allocating new open transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:52:36,545 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testWriteTransactionWithMultipleShards: allocating new open transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:52:36,545 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:52:36,545 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 ready 01:52:36,546 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: transitioned member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 to ready 01:52:36,546 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating canCommit 01:52:36,546 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:52:36,546 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:52:36,546 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 ready 01:52:36,546 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 validated 01:52:36,546 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:52:36,546 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: transitioned member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 to ready 01:52:36,546 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating canCommit 01:52:36,546 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed canCommit 01:52:36,546 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:52:36,546 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 validated 01:52:36,546 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 01:52:36,546 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed canCommit 01:52:36,546 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0}} 01:52:36,546 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=910871579033, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0}} 01:52:36,546 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 01:52:36,547 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0}} 01:52:36,547 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=910871667725, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0}} 01:52:36,547 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} canCommit completed 01:52:36,547 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 01:52:36,547 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1823442556]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@5ce69353 01:52:36,547 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} 01:52:36,547 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} allocated sequence 1 01:52:36,547 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1823442556]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x0000000801303580@816fb1d 01:52:36,547 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-804479436], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:36,547 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6187feb0 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1823442556]}, enqueuedTicks=910877721945} 01:52:36,547 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 01:52:36,547 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1823442556]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1223/0x0000000801304a88@6611250a 01:52:36,547 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} 01:52:36,547 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} allocated sequence 1 01:52:36,547 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1823442556]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1241/0x0000000801303580@5774a039 01:52:36,547 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1726422422], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:52:36,547 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@60730a61 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1823442556]}, enqueuedTicks=910877859698} 01:52:36,547 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating preCommit 01:52:36,547 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:52:36,547 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating preCommit 01:52:36,547 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:52:36,547 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 candidate ready 01:52:36,547 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1: canCommit - candidate rootPath: / 01:52:36,547 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 candidate ready 01:52:36,547 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2: canCommit - candidate rootPath: / 01:52:36,547 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1: canCommit - messages: [] 01:52:36,548 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 01:52:36,548 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 prepared 01:52:36,548 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:52:36,548 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2: canCommit - messages: [] 01:52:36,548 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed preCommit 01:52:36,548 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2: preCommit - successfulFromPrevious: [] 01:52:36,548 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 prepared 01:52:36,548 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 01:52:36,548 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed preCommit 01:52:36,548 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1823442556]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1}} 01:52:36,548 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1823442556]}, enqueuedTicks=910877721945, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1}} 01:52:36,548 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:52:36,548 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1823442556]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1}} 01:52:36,548 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1823442556]}, enqueuedTicks=910877859698, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1}} 01:52:36,548 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 01:52:36,548 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 01:52:36,548 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1823442556]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@91076d 01:52:36,548 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} 01:52:36,548 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} allocated sequence 2 01:52:36,548 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1823442556]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@91076d 01:52:36,548 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-804479436], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:36,548 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6187feb0 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1823442556]}, enqueuedTicks=910879476239} 01:52:36,548 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 2 01:52:36,548 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1823442556]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@6f474b00 01:52:36,548 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} 01:52:36,549 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} allocated sequence 2 01:52:36,549 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1823442556]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1231/0x00000008013078c0@6f474b00 01:52:36,549 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1726422422], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:52:36,549 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@60730a61 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1823442556]}, enqueuedTicks=910879580811} 01:52:36,550 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating commit 01:52:36,550 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:52:36,550 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating commit 01:52:36,550 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:52:36,550 PM [cluster-test-shard-dispatcher-15] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 176 01:52:36,550 PM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 180 01:52:36,550 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 01:52:36,550 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 01:52:36,550 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:52:36,550 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:52:36,550 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:52:36,550 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 01:52:36,550 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:52:36,550 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 01:52:36,550 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:52:36,550 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}]]] 01:52:36,550 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testWriteTransactionWithMultipleShards 01:52:36,550 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 submitted to persistence 01:52:36,550 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 01:52:36,550 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 submitted to persistence 01:52:36,551 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 01:52:36,551 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:52:36,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}]]] 01:52:36,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:52:36,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:52:36,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=176} 01:52:36,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 01:52:36,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Log size is now 1 01:52:36,551 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 01:52:36,551 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:52:36,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:52:36,551 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 01:52:36,551 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=180} 01:52:36,551 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:52:36,551 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Log size is now 1 01:52:36,551 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:52:36,551 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 01:52:36,551 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 01:52:36,552 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithMultipleShards status sync done true 01:52:36,552 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testWriteTransactionWithMultipleShards status sync done true 01:52:36,552 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:52:36,552 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 01:52:36,552 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:52:36,552 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 01:52:36,552 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 01:52:36,552 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:52:36,552 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 01:52:36,552 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=176} 01:52:36,552 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:52:36,552 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=180} 01:52:36,552 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:52:36,552 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:52:36,552 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-1-shard-cars-testWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4bdb9063}, purged=MutableUnsignedLongSet{size=0}} 01:52:36,552 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendMetadata - member-1-shard-people-testWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@21b72579}, purged=MutableUnsignedLongSet{size=0}} 01:52:36,552 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 01:52:36,552 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:52:36,552 PM [cluster-test-shard-dispatcher-15] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2: commit - successfulFromPrevious: [] 01:52:36,552 PM [cluster-test-shard-dispatcher-15] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 01:52:36,552 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-1651707914] with name member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher 01:52:36,552 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-1651707914] 01:52:36,552 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:52:36,552 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:52:36,552 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:52:36,552 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-1651707914] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher: 102.2 μs 01:52:36,553 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 01:52:36,553 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:52:36,553 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1: commit - successfulFromPrevious: [] 01:52:36,553 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 01:52:36,553 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#148006925] with name member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#148006925] 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:52:36,553 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 01:52:36,553 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher: 46.10 μs 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#148006925] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1823442556]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2}} 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1823442556]}, enqueuedTicks=910879580811, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2}} 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} doCommit completed 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6ff31428 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR}: initiating purge 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 3 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1823442556]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@7885aade 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED}: initiating purge 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} allocated sequence 3 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1823442556]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@2183e90f 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1726422422], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@60730a61 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1823442556]}, enqueuedTicks=910884263447} 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1823442556]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2}} 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1823442556]}, enqueuedTicks=910879476239, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2}} 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 01:52:36,553 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@770d3790: creating a new snapshot member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@639a5e7e completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 01:52:36,553 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1823442556]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@4ae3d1ed 01:52:36,554 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} 01:52:36,554 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:52:36,554 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED}: initiating purge 01:52:36,554 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} allocated sequence 0 01:52:36,554 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} allocated sequence 3 01:52:36,554 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1823442556]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1232/0x0000000801307af8@1fe54135 01:52:36,554 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@799de166 01:52:36,554 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-804479436], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:36,554 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6187feb0 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1823442556]}, enqueuedTicks=910884519463} 01:52:36,554 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6187feb0 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=910884621146} 01:52:36,554 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Commit index set to 0 01:52:36,554 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Commit index set to 0 01:52:36,554 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:52:36,554 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 01:52:36,554 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:52:36,554 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=180} 01:52:36,554 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 01:52:36,554 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 01:52:36,554 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 01:52:36,554 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=176} 01:52:36,554 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:52:36,554 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:52:36,554 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-people-testWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@430eabda}, purged=MutableUnsignedLongSet{size=0}} 01:52:36,554 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@67eb042}, purged=MutableUnsignedLongSet{size=0}} 01:52:36,554 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 01:52:36,554 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 01:52:36,554 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:52:36,554 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:52:36,554 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#713636612] with name member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher 01:52:36,554 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-1671700693] with name member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher 01:52:36,554 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:52:36,554 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 01:52:36,555 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-1671700693] 01:52:36,555 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#713636612] 01:52:36,555 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:52:36,555 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 01:52:36,555 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher: 50.67 μs 01:52:36,555 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher: 53.19 μs 01:52:36,555 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-1671700693] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:36,555 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#713636612] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 01:52:36,559 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: purging transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:52:36,559 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 01:52:36,559 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: purging transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:52:36,559 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 01:52:36,559 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 01:52:36,559 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:52:36,559 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 01:52:36,559 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 01:52:36,560 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testWriteTransactionWithMultipleShards 01:52:36,560 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}]]] 01:52:36,560 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 01:52:36,560 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}]]] 01:52:36,560 PM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithMultipleShards: allocating new open snapshot member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1 01:52:36,560 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithMultipleShards_slicer-id-727: slice: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 01:52:36,560 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-1-shard-cars-testWriteTransactionWithMultipleShards_slicer-id-727: Message size: 332 bytes, total slices to send: 1 01:52:36,560 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithMultipleShards_slicer-id-727: Message does not need to be sliced - sending original message 01:52:36,560 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 01:52:36,560 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 01:52:36,560 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}]]] 01:52:36,560 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:52:36,560 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}]]] 01:52:36,560 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:52:36,560 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 01:52:36,560 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68} 01:52:36,560 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 01:52:36,560 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 01:52:36,560 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68} 01:52:36,560 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 01:52:36,560 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Log size is now 2 01:52:36,560 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Log size is now 2 01:52:36,560 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:52:36,561 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 01:52:36,561 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 01:52:36,561 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=910884621146, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 01:52:36,561 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0} 01:52:36,561 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 01:52:36,561 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 01:52:36,561 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 01:52:36,561 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} allocated sequence 0 01:52:36,561 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1192/0x00000008012d45a0@78af6ebc 01:52:36,561 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@60730a61 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people}, enqueuedTicks=910892081461} 01:52:36,561 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:52:36,561 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:52:36,561 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 01:52:36,561 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 01:52:36,561 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:52:36,561 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 01:52:36,561 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68} 01:52:36,561 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 01:52:36,561 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: replication of member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@54e0ead 01:52:36,561 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:52:36,561 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68} 01:52:36,561 PM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testWriteTransactionWithMultipleShards: finished purging transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:52:36,561 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: replication of member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2244/0x00000008016b69b0@232706d7 01:52:36,561 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithMultipleShards: finished purging transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:52:36,561 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 01:52:36,561 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:52:36,561 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:52:36,561 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 01:52:36,561 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:52:36,561 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:52:36,562 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:52:36,562 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:52:36,562 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:52:36,562 PM [cluster-test-shard-dispatcher-15] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testWriteTransactionWithMultipleShards: allocating new open snapshot member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2 01:52:36,562 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:52:36,562 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-people-testWriteTransactionWithMultipleShards_slicer-id-728: slice: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, message: Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 01:52:36,562 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-1-shard-people-testWriteTransactionWithMultipleShards_slicer-id-728: Message size: 337 bytes, total slices to send: 1 01:52:36,562 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-people-testWriteTransactionWithMultipleShards_slicer-id-728: Message does not need to be sliced - sending original message 01:52:36,562 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1823442556]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} 01:52:36,562 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1823442556]}, enqueuedTicks=910884263447, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} 01:52:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Commit index set to 1 01:52:36,563 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE}: purge completed 01:52:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:52:36,563 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1b256c2 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} 01:52:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 01:52:36,563 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR}: purge completed 01:52:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:52:36,563 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6ff31428 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 01:52:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68} 01:52:36,563 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1823442556]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} 01:52:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 01:52:36,563 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1823442556]}, enqueuedTicks=910884519463, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} 01:52:36,563 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE}: purge completed 01:52:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 01:52:36,563 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@352da49a purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} 01:52:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:52:36,563 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 01:52:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:52:36,563 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@639a5e7e purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 01:52:36,563 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people} with Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 01:52:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Commit index set to 1 01:52:36,563 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1823442556], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people}, enqueuedTicks=910892081461, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 01:52:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 01:52:36,563 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0} 01:52:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 01:52:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 01:52:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68} 01:52:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 01:52:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 01:52:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 01:52:36,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 01:52:36,563 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:52:36,563 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testWriteTransactionWithMultipleShards 01:52:36,563 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testWriteTransactionWithMultipleShards 01:52:36,563 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithMultipleShards - waiting on 2 shards 01:52:36,563 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onShutDown 01:52:36,563 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Initiating leader transfer 01:52:36,563 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onShutDown 01:52:36,563 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@66c0e38 01:52:36,563 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Initiating leader transfer 01:52:36,563 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@66c0e38 01:52:36,563 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 01:52:36,563 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:52:36,563 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@71139e74 01:52:36,563 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:52:36,563 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@71139e74 01:52:36,563 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 01:52:36,563 PM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:52:36,563 PM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 01:52:36,563 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:52:36,563 PM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:36,563 PM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 01:52:36,563 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:52:36,564 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:52:36,564 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 01:52:36,564 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:52:36,564 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 01:52:36,564 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:52:36,564 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 01:52:36,564 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Leader transfer complete 01:52:36,564 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Leader transfer complete 01:52:36,564 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards] 01:52:36,565 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards] 01:52:36,565 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:52:36,565 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 01:52:36,565 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 01:52:36,565 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 01:52:36,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 01:52:36,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@60730a61 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@71537c72, splicing queue 01:52:36,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@60730a61 queue spliced 0 messages 01:52:36,565 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1726422422], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1726422422], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 01:52:36,565 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: refreshing backend for shard 2 01:52:36,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1726422422], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 01:52:36,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 01:52:36,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 01:52:36,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-2115265394] 01:52:36,565 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 01:52:36,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 01:52:36,565 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:52:36,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6187feb0 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@54f3cd58, splicing queue 01:52:36,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6187feb0 queue spliced 0 messages 01:52:36,565 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-804479436], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-804479436], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 01:52:36,565 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: refreshing backend for shard 1 01:52:36,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-804479436], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 01:52:36,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 01:52:36,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 01:52:36,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-2115265394] 01:52:36,565 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 01:52:36,565 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 01:52:36,974 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 01:52:36,974 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-804479436]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 01:52:36,974 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-1-shard-people-testWriteTransactionWithMultipleShards 01:52:36,974 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Stopping Shard member-1-shard-cars-testWriteTransactionWithMultipleShards 01:52:36,974 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader)_slicer-id-732: Closing 01:52:36,974 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader)_slicer-id-733: Closing 01:52:36,975 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:52:36,975 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:52:36,975 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithMultipleShards 01:52:36,975 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 01:52:36,975 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 01:52:36,975 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 01:52:36,978 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 01:52:36,978 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:36,978 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/11/2024 13:52:36.982] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1103)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:52:36.983] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1103)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:52:36.983] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1103)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/11/2024 13:52:36.983] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1103)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:36,996 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 01:52:36,997 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithMultipleShards 01:52:36,997 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Closing 01:52:36,997 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-people-testWriteTransactionWithMultipleShards 01:52:36,997 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Closing 01:52:36,997 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:52:36,997 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 01:52:36,997 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithMultipleShards 01:52:36,998 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:36,998 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:38,0 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-6705261226330559508] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:52:38,10 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-3281023276872059784] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 01:52:38,14 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:38,15 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 01:52:38,16 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:52:38.017] [cluster-test-akka.actor.internal-dispatcher-5] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 01:52:38,24 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 01:52:38,24 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 01:52:38,26 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/11/2024 13:52:38.027] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config